CodeChef
  • PRACTICE
    • Easy
    • Medium
    • Hard
    • Challenge
    • Peer
  • COMPETE
  • DISCUSS
    • Wiki
    • Forums
    • Blog
    • Twitter
  • COMMUNITY
    • CodeChef Meetups
    • Campus Chapters
    • Host Your Contest
    • User Groups
    • CodeChef TechTalks
  • HELP
    • Frequently Asked Questions
    • FAQ for Problem Setters
    • Ranks
    • Tutorials
  • ABOUT
    • About CodeChef
    • Team CodeChef
    • Press Room
    • CEO's Corner
    • About Directi

The new rating system

Posted by Aniruddha on August 28th, 2009 Filed in About, Announcement, Features View Comments

Greetings everyone!

We are happy to introduce a rating system for Codechef based on the contests we have every month. In brief, the rating system will have the following features.

1. Every user will have a rating. The user with the highest rating will be ranked 1, the user with the next highest rating will be ranked 2 and so on.

2. The ratings for the users will depend only on how they perform in the contests and not how they perform in the practice section.

3. Every contest problem will get a particular value based on the number of users solving that problem. The maximum value for a problem tends to the value 5 and the minimum value for a problem tends to 1. So, every user is guaranteed at-least 1 point for solving a problem. The selection of the value 5 was to give sufficient room for the top performers to increase their rating. Selection of a smaller range of values would let those participants solving only easy problems and participating frequently overtake those not participating frequently but performing exceptionally well when they do. Suppose the total number of users who have solved a particular problem is PS and the total number of users who have participated in the contest is TS, then the point value for a particular problem is 5 – 4*PS/TS. So, if there are more users successfully solving a problem, it’s value will be less and vice versa. The Challenge problem point value is calculated by taking the product of the score for the challenge problem with the maximum rating a contest problem can have i.e 5.

4. Suppose a user has solved P binary problems and 1 challenge problem and at the end of the contest, these problems have the values V1, V2, .. ,Vp then the total rating score S of the user for that contest will be V1+V2+…+Vp + 5 * (score of the user for the challenge problem).

5. The new rating of the user will be old_rating+S.

Also, every user’s profile page will now have a rating graph along with the current rating and rank. Your feedback in this regard is much appreciated :)

Regards,
Aniruddha.

  • Share/Bookmark
  • Sanket Shah

    Makes the competition even more cooler!

  • Sanket Shah

    Makes the competition even more cooler!

  • Aniruddha

    :) Sure it does.

  • http://web.iiit.ac.in/~anshuman Anshuman Singh

    Hey, couldn’t you normalize it to be something per contest with some weightage on the number of contests participated in. There is a lot to lose for people who miss out on even a single contest.

  • http://web.iiit.ac.in/~anshuman Anshuman Singh

    Hey, couldn’t you normalize it to be something per contest with some weightage on the number of contests participated in. There is a lot to lose for people who miss out on even a single contest.

  • Keshav Dhandhania

    Exactly. Please make sure that the ratings are not biased towards a person who participates in more contests.

  • Keshav Dhandhania

    Exactly. Please make sure that the ratings are not biased towards a person who participates in more contests.

  • Keshav Dhandhania

    Secondly, I think that the events which took place before the rating system was formulated, should not influence the rating.

    Thirdly, Is there a link where we can see the list of usernames, sorted by rank/rating.

  • Keshav Dhandhania

    Secondly, I think that the events which took place before the rating system was formulated, should not influence the rating.

    Thirdly, Is there a link where we can see the list of usernames, sorted by rank/rating.

  • Josh Metzler

    Can the tie-breaker problem give a variable number of rating points related to the rank achieved on the problem? I ask this because often submitting a solution that does basically nothing can “solve” the tie-breaker, but to achieve a top score can take a lot of work. It could be something as simple as multiplying the tie-breaker’s point value by the score achieved on it.

  • Josh Metzler

    Can the tie-breaker problem give a variable number of rating points related to the rank achieved on the problem? I ask this because often submitting a solution that does basically nothing can “solve” the tie-breaker, but to achieve a top score can take a lot of work. It could be something as simple as multiplying the tie-breaker’s point value by the score achieved on it.

  • Aniruddha

    @Josh Forgot to mention that. The tie breaker is judged exactly how you described.

  • Aniruddha

    @Josh Forgot to mention that. The tie breaker is judged exactly how you described.

  • Vaibhav

    I agree with Anshuman and Keshav that we need a normalized scoring but I’d like to point out that even that system isn’t going to be perfect. In TopCoder for instance, a new account holder could do really well in just one SRM and not participate in anything further, and have a rating as high as a late blue or early yellow. Meaning that just as people missing out on one contest is a concern in the existing system, people not attending many contests will be a concern for the normalised system. Perhaps a balance amongst the two could be struck (minimum number of contests to participate in before one is rated etc)?

  • Vaibhav

    I agree with Anshuman and Keshav that we need a normalized scoring but I’d like to point out that even that system isn’t going to be perfect. In TopCoder for instance, a new account holder could do really well in just one SRM and not participate in anything further, and have a rating as high as a late blue or early yellow. Meaning that just as people missing out on one contest is a concern in the existing system, people not attending many contests will be a concern for the normalised system. Perhaps a balance amongst the two could be struck (minimum number of contests to participate in before one is rated etc)?

  • Aniruddha

    There are drawbacks to any system as Vaibhav correctly pointed out. In the normalized version, if someone were to perform really well in one of the contests and did not find time to participate actively in some other, his rating would go down. Also, in the current system, if someone misses one contest, his / her rank won’t go down by a lot if he / she has performed really well previously. We want to encourage people to participate in more number of contests at the same time.

  • Aniruddha

    There are drawbacks to any system as Vaibhav correctly pointed out. In the normalized version, if someone were to perform really well in one of the contests and did not find time to participate actively in some other, his rating would go down. Also, in the current system, if someone misses one contest, his / her rank won’t go down by a lot if he / she has performed really well previously. We want to encourage people to participate in more number of contests at the same time.

  • Pratik

    Is there any ranks page?
    I see the ratings and ranks put up on the profile.

  • Pratik

    Is there any ranks page?
    I see the ratings and ranks put up on the profile.

  • Hemant

    I think the contest before the rating system should not be included.

    A normalization would be there , the number of event doesn’t matter the performance matter, also why practice problems are taken into account , anybody can submit the duplicated code for them.

  • Hemant

    I think the contest before the rating system should not be included.

    A normalization would be there , the number of event doesn’t matter the performance matter, also why practice problems are taken into account , anybody can submit the duplicated code for them.

  • Aniruddha

    Practice problems are not considered while calculating the rating.

  • Aniruddha

    Practice problems are not considered while calculating the rating.

  • Hemant

    How you account for new user , is they have to practice really hard compare to their earlier counter part. Also they can’t match other people according to new system.

    One more bug : their is no indication of what is the highest rating (overall) , so that graph should be made according to that.

  • Hemant

    How you account for new user , is they have to practice really hard compare to their earlier counter part. Also they can’t match other people according to new system.

    One more bug : their is no indication of what is the highest rating (overall) , so that graph should be made according to that.

  • Neo Coder

    I think (1-PS/TS) would have been a good value for a correct solution to a problem. I am not sure how 5-4*PS/TS was come up!

    I just ran a sample for typical easy but many problem solving coder to a few but difficult problem solving coder and i found your formula would not help the later much as you claim. Whats the median you are looking at between regular easy problem solver and rare but tough problem solver?

  • Neo Coder

    I think (1-PS/TS) would have been a good value for a correct solution to a problem. I am not sure how 5-4*PS/TS was come up!

    I just ran a sample for typical easy but many problem solving coder to a few but difficult problem solving coder and i found your formula would not help the later much as you claim. Whats the median you are looking at between regular easy problem solver and rare but tough problem solver?

  • Aniruddha

    The current formula would help the latter case more than 1-PS/TS. The choice of the number 5 was arbitrary.

  • Aniruddha

    The current formula would help the latter case more than 1-PS/TS. The choice of the number 5 was arbitrary.

  • anupam

    Some of the practice problems are also very tough. Why cant the admins include performance in practice problems to determine grade. Because it is also possible that a person may succeed in solving contest problem but fails to solve some practice problems. Thus performance in practice problems might also play its role in determining rank of the person.

  • anupam

    Some of the practice problems are also very tough. Why cant the admins include performance in practice problems to determine grade. Because it is also possible that a person may succeed in solving contest problem but fails to solve some practice problems. Thus performance in practice problems might also play its role in determining rank of the person.

  • Aniruddha

    One of the major problems with this is that a lot of the practice problems have their solutions publicly visible. Also, there is a lot of discussion related to the practice problems in the forums and on the problem page.

  • anupam

    well I have seen solutions to some contest problems are also visible.secondly people also discuss about contest problems in codechef. If we visit the site http://acm.timus.ru we will find all practice problems determine worldwide rank along with various contest problems.People are allowed to discuss algorithms for solving a problem but are not allowed to see the code. admins of the site remove codes but do not remove algorithms (if codes are posted in the forum by mischief) Infact discussion of algo increases your knowledge but its very difficult to understand cryptic code. Thus I feel viewing solutions made by other person does not help much. So it does’nt really matter if codes are visible or not.I feel each problem should have a rating given say within scale of 1 – 10. The problems whose solutions are avaible should have lowest rating attached to it. Thus people only copying code cannot make it to the top ranks. Well this is only a suggestion it remains to the admins to decide what to do.

  • anupam

    well I have seen solutions to some contest problems are also visible.secondly people also discuss about contest problems in codechef. If we visit the site http://acm.timus.ru we will find all practice problems determine worldwide rank along with various contest problems.People are allowed to discuss algorithms for solving a problem but are not allowed to see the code. admins of the site remove codes but do not remove algorithms (if codes are posted in the forum by mischief) Infact discussion of algo increases your knowledge but its very difficult to understand cryptic code. Thus I feel viewing solutions made by other person does not help much. So it does’nt really matter if codes are visible or not.I feel each problem should have a rating given say within scale of 1 – 10. The problems whose solutions are avaible should have lowest rating attached to it. Thus people only copying code cannot make it to the top ranks. Well this is only a suggestion it remains to the admins to decide what to do.

  • Sanket Shah

    May be decrease of the rating on not participating..?
    So instead of same rating it decreases according to some curve.

  • Sanket Shah

    May be decrease of the rating on not participating..?
    So instead of same rating it decreases according to some curve.

  • TEST

    Testing if this is working

  • TEST

    Testing if this is working

  • Pradeep

    5-4*PS/TS is equivalent to 1-0.8*PS/TS. It basically assures a person of getting a positive score (>1) for a problem, unlike in 1-PS/TS, where if everyone solves it, everyone gets a zero.

  • Pradeep

    5-4*PS/TS is equivalent to 1-0.8*PS/TS. It basically assures a person of getting a positive score (>1) for a problem, unlike in 1-PS/TS, where if everyone solves it, everyone gets a zero.

  • avinash kumar

    when my rating will come?

  • avinash kumar

    when my rating will come?

  • http://tessellationsoftechnology.wordpress.com/ ishan

    I think there should be a separate category of rating for practice problems to encourage beginners who wants to practise for the time being and are not quite ready for contests yet.

  • http://tessellationsoftechnology.wordpress.com/ ishan

    I think there should be a separate category of rating for practice problems to encourage beginners who wants to practise for the time being and are not quite ready for contests yet.

  • Vinayak Garg

    Hi why am i getting 0 rating. July 2010 ranking is showing my score as 1.

  • Vinayak Garg

    Hi why am i getting 0 rating. July 2010 ranking is showing my score as 1.

  • Vinayak Garg

    Hi why am i getting 0 rating. July 2010 ranking is showing my score as 1.

  • govardhan gopi

    Does not cook-off contributes in rating ?? I did not get any change in rating after solving two problems in yesterday’s cook-off ( August ) … !!

  • govardhan gopi

    Does not cook-off contributes in rating ?? I did not get any change in rating after solving two problems in yesterday’s cook-off ( August ) … !!

  • Anup

    @goverdhan: the cook off is a shorter format and hence the same rating does not apply here. It will unfair to club the 2 formats together. Once we start having cook-off’s on a more regular basis, we shall incorporate an independent rating system for the same.

  • Tamilanban3

    when will come my rating

  • Rahul2025

    plz introduce rating for practice problems also….
    This will definitely encourage the beginners, who are not ready for contests yet…

  • Tejwinder Singh

    @admin in ranking what is long nd short
    @admin:disqus     and the practice problems are counted for ranking

  • http://www.codechef.com CodeChef

    We have two contests at CodeChef per month. The 10 day long contest that runs from the 1st to the 11th of every month and the 2.5 hours running short contest that runs on the second last Sunday of every month. Each one of them has an individual rating system that gives a user two different rankings. Also the practice problems are not counted for ranking an individual as of now.

  • Tejwinder91

    thank you :)

  • Nikhilkarnwal

    I want to know why my rating decreases after august cook-off as i have solved one question.. 

  • http://www.cse.iitb.ac.in/~teja Sai Teja Pratap

    @admin:disqus , How is value of TS( total number of users who have participated in the contest) calculated?

  • vinayneekhra

    The country flag on the profile page, will look better if it is smaller in size.

blog comments powered by Disqus

Recent Posts

  • ACRush eclipsed
  • … and we meet!
  • Progress report of April contests
  • Moving into a New Kitchen.
  • The color and mischief of March contests

Categories

  • About (8)
  • ACM ICPC (9)
  • Announcement (119)
  • Campus Chapters (6)
  • College Contests (8)
  • Contests (126)
  • Events (23)
  • FAQ (1)
  • Features (34)
  • Languages (1)
  • Meetup (5)
  • Open Source (1)
  • Practice Problems (7)
  • Prizes (17)
  • Problems (5)
  • Programmer of the Month (27)
  • Tech Talks (6)
  • Tutorials (15)
  • Winners (80)

Recent Comments

  • CodeChef on ACRush eclipsed
  • CodeChef on ACRush eclipsed
  • random123 on ACRush eclipsed
  • Sagar Malhotra on Moving into a New Kitchen.
  • Guest on Moving into a New Kitchen.

Recent Pictures

Blogroll

  • Documentation
  • Plugins
  • Suggest Ideas
  • Support Forum
  • Themes
  • WordPress Blog
  • WordPress Planet

Archives

  • May 2013
  • April 2013
  • March 2013
  • February 2013
  • January 2013
  • December 2012
  • October 2012
  • September 2012
  • August 2012
  • July 2012
  • June 2012
  • May 2012
  • April 2012
  • March 2012
  • February 2012
  • January 2012
  • December 2011
  • November 2011
  • October 2011
  • September 2011
  • August 2011
  • July 2011
  • June 2011
  • May 2011
  • April 2011
  • March 2011
  • February 2011
  • January 2011
  • December 2010
  • November 2010
  • October 2010
  • September 2010
  • August 2010
  • July 2010
  • June 2010
  • May 2010
  • April 2010
  • March 2010
  • February 2010
  • January 2010
  • December 2009
  • November 2009
  • October 2009
  • September 2009
  • August 2009
  • July 2009
  • June 2009
  • May 2009
  • April 2009
  • March 2009
  • February 2009
  • January 2009

Company Blogs

  • Directi
  • .pw Corp Blog
  • CEOs Blog

Careers@Directi


  • About CodeChef
  • About Directi
  • CEO's Corner
  • CodeChef Campus Chapters
  • Blogger Community Program
  • User Group Outreach Program

© 2009, Directi Group. All Rights Reserved.

Sponsors