CodeChef
  • PRACTICE
    • Easy
    • Medium
    • Hard
    • Challenge
    • Peer
  • COMPETE
  • DISCUSS
    • Wiki
    • Forums
    • Blog
    • Twitter
  • COMMUNITY
    • CodeChef Meetups
    • Campus Chapters
    • Host Your Contest
    • User Groups
    • CodeChef TechTalks
  • HELP
    • Frequently Asked Questions
    • FAQ for Problem Setters
    • Ranks
    • Tutorials
  • ABOUT
    • About CodeChef
    • Team CodeChef
    • Press Room
    • CEO's Corner
    • About Directi

Important Rule Change for July Contest

Posted by The Chef on June 30th, 2009 Filed in About, Contests View Comments

Chocolate Shakes,
We’ve had some complaints in the last contest about participants reverse engineering test data. While this isn’t necessarily “wrong,” in keeping with the spirit of finding the best solutions to the problems given, we’ve instituted a small rule change. This time around, the challenge problem will be rejudged at the end of the contest, with entirely new test data.

We considered other approaches for all problems including:
* Rejudging all solutions at the end of the contest with new test data (not just the challenge problem)
* Capping the number of submissions per user per day
* Allowing users to only submit solutions at certain time intervals (i.e. one solution per five min)

In the end we have done our best to come up with better, bigger and more random test cases and are holding off on these other change (for now). If you believe there are additional ways to improve the contest format, please let us know.

Cheers,
Amit (The Chef)

p.s. if you wanna see how pretty I am, check out our new Twitter background.

p.p.s don’t forget our first DesignChef competition along starts tomorrow, help us out… spread the word.

  • Share/Bookmark
  • Infinite_logic

    Hi Chef,

    I know that you really want to make it fair but if you are re-judging the solutions , what about those solutions which will fail the new test cases but pass the old, in that case it will not give the participants to re-submit their codes, in case it is wrong. While in the online competition one can submit as long as his solution is wrong.

    Why the cap ?? Most of the programmers here are college students and will probably not be coding all days of the week. So those who will feel like coding on one or two days only are gonna suffer badly…anyway I have no objections to this.

    Again the cap at one solution per five minutes… If you are already implementing the cap per day then why the need for another cap ?? I mean a person will have to fruitlessly wait for the 300 seconds to pass before he actually submit again. Now if there is a compilation error or some stupid mistake in the code and one wants to resubmit again.. I am sure he/she will be damn..pissed ..

    I fully feel that instead of days the contest could be reduced to hours just like topcoder and that way it will be much better otherwise I have a feeling that some of the solutions are not entirely the coder’s own work.. but I would rather call it collaboration ..and it’s gotta be fair right Chef??

  • Infinite_logic

    Hi Chef,

    I know that you really want to make it fair but if you are re-judging the solutions , what about those solutions which will fail the new test cases but pass the old, in that case it will not give the participants to re-submit their codes, in case it is wrong. While in the online competition one can submit as long as his solution is wrong.

    Why the cap ?? Most of the programmers here are college students and will probably not be coding all days of the week. So those who will feel like coding on one or two days only are gonna suffer badly…anyway I have no objections to this.

    Again the cap at one solution per five minutes… If you are already implementing the cap per day then why the need for another cap ?? I mean a person will have to fruitlessly wait for the 300 seconds to pass before he actually submit again. Now if there is a compilation error or some stupid mistake in the code and one wants to resubmit again.. I am sure he/she will be damn..pissed ..

    I fully feel that instead of days the contest could be reduced to hours just like topcoder and that way it will be much better otherwise I have a feeling that some of the solutions are not entirely the coder’s own work.. but I would rather call it collaboration ..and it’s gotta be fair right Chef??

  • http://www.codechef.com The Chef

    Hi Mohit,
    Please note that only the Challenge problem will be rejudged. The other suggestions will not be implemented. The goal is to keep within the spirit of the competition (to find the best solutions to problems) and not reverse engineer test data.
    Thanks,
    Chef

  • http://www.codechef.com The Chef

    Hi Mohit,
    Please note that only the Challenge problem will be rejudged. The other suggestions will not be implemented. The goal is to keep within the spirit of the competition (to find the best solutions to problems) and not reverse engineer test data.
    Thanks,
    Chef

  • Saurabh Sethia

    In algorithmic contests the time limits should be such that most programs with the right complexity pass. Solutions that use standard data structures like set and map from standard template library of C++ should pass because otherwise you are asking the programmers to reinvent the wheel. Similarly using cin/cout instead of scanf/printf should not lead to TLE. You can always have at least one huge test case such that you can tolerate larger multiplicative and additive constants. Otherwise the focus gets removed from algorithms and we have to focus on optimizations that discourage use of standard libraries. Using time limits that tolerate standard library features will lead to many elegant solutions that currently get rejected.

  • Saurabh Sethia

    In algorithmic contests the time limits should be such that most programs with the right complexity pass. Solutions that use standard data structures like set and map from standard template library of C++ should pass because otherwise you are asking the programmers to reinvent the wheel. Similarly using cin/cout instead of scanf/printf should not lead to TLE. You can always have at least one huge test case such that you can tolerate larger multiplicative and additive constants. Otherwise the focus gets removed from algorithms and we have to focus on optimizations that discourage use of standard libraries. Using time limits that tolerate standard library features will lead to many elegant solutions that currently get rejected.

  • https://totobogy.thoughtbubblez.com Tushar

    I second the point raised by Saurabh above. In real world software development, performance is a very important but not the only driving factor. Design and elegance of solutions is equally important and code reuse by means of standard or previously developed libraries is not only encouraged but also essential to keep things sane. You don’t go and code-up a new hash-table implementation unless it’s has been nailed down as ‘the’ cause of performance bottleneck in the system by extensive, targeted testing (which is rare). Most of the time, it is the design / archtitecture of the system (or algorithm) which is the culprit not some corner-case idiosyncracies.

  • https://totobogy.thoughtbubblez.com Tushar

    I second the point raised by Saurabh above. In real world software development, performance is a very important but not the only driving factor. Design and elegance of solutions is equally important and code reuse by means of standard or previously developed libraries is not only encouraged but also essential to keep things sane. You don’t go and code-up a new hash-table implementation unless it’s has been nailed down as ‘the’ cause of performance bottleneck in the system by extensive, targeted testing (which is rare). Most of the time, it is the design / archtitecture of the system (or algorithm) which is the culprit not some corner-case idiosyncracies.

  • Prunthaban

    @Tushar Real world software development and algorithm contest are too different things. PERIOD.

  • Prunthaban

    @Tushar Real world software development and algorithm contest are too different things. PERIOD.

  • http://ajayfromiiit.wordpress.com Ajay Somani

    I believe that the rejudge is a brillient idea. I guess all other solutions mentioned are not good. The rejudge should only be done on tie breaker and also the other limits dont make much sense to me, the waiting time could lead to an unsatisfactory experience from the website :) .

    I havent checked out this month’s problems yet. I hope they have good test data and should have tiem limit which stresses more on the algorithm rather than low level optimizations unlike problem D4 of last month.

  • http://ajayfromiiit.wordpress.com Ajay Somani

    I believe that the rejudge is a brillient idea. I guess all other solutions mentioned are not good. The rejudge should only be done on tie breaker and also the other limits dont make much sense to me, the waiting time could lead to an unsatisfactory experience from the website :) .

    I havent checked out this month’s problems yet. I hope they have good test data and should have tiem limit which stresses more on the algorithm rather than low level optimizations unlike problem D4 of last month.

  • pharaoh

    @Prunthaban

    They need not be different. COMMA

  • pharaoh

    @Prunthaban

    They need not be different. COMMA

  • Tomek Czajka

    I think some kind of a limit on the number of submissions (per day, per hour, or total) would be preferable to rejudging on new test data.

    Some reasons:
    - the scoreboard would be “real”
    - accepted solutions may break during rejudging, especially if only the last one is taken into account; for example, one might use too much memory (the limits are not clear)

    The “input data mining” problem wouldn’t be so bad if the number of submissions is capped. In a problem like the one in July’09 contest, it’s hardly a problem at all, because the data is so large.

    It might actually be good to allow some research on input data statistics – otherwise the contest may hinge in *guessing* what data the judges are going to provide, in order to make a good solution adapted for that kind of data.

    Without rejudging, contestants will do some input stats and that’s quite fair (e.g. test distribution of input sizes).

  • Tomek Czajka

    I think some kind of a limit on the number of submissions (per day, per hour, or total) would be preferable to rejudging on new test data.

    Some reasons:
    - the scoreboard would be “real”
    - accepted solutions may break during rejudging, especially if only the last one is taken into account; for example, one might use too much memory (the limits are not clear)

    The “input data mining” problem wouldn’t be so bad if the number of submissions is capped. In a problem like the one in July’09 contest, it’s hardly a problem at all, because the data is so large.

    It might actually be good to allow some research on input data statistics – otherwise the contest may hinge in *guessing* what data the judges are going to provide, in order to make a good solution adapted for that kind of data.

    Without rejudging, contestants will do some input stats and that’s quite fair (e.g. test distribution of input sizes).

blog comments powered by Disqus

Recent Posts

  • The February Long Challenge has a Surprise in store!
  • Gennady’s blitz·krieg!
  • An overwhelming participation for IOPC2012 :)
  • IIT-Kanpur along with CodeChef presents Techkriti-IOPC 2012
  • A “tie’ing” start to the year

Categories

  • About (15)
  • Announcement (90)
  • Campus Chapters (4)
  • Contests (94)
  • Events (17)
  • Features (19)
  • Meetup (4)
  • Open Source (1)
  • Practice Problems (6)
  • Prizes (32)
  • Programmer of the Month (27)
  • Tech Talks (2)
  • Tutorials (24)
  • Winners (44)

Recent Comments

  • Anish_7766 on The February Long Challenge has a Surprise in store!
  • Berita Gue on October’s Programmer of Month: Hiroto Sekido
  • pERCY on The February Long Challenge has a Surprise in store!
  • Vivek5402 on The February Long Challenge has a Surprise in store!
  • Vivek on The February Long Challenge has a Surprise in store!

Recent Pictures

Blogroll

  • Documentation
  • Plugins
  • Suggest Ideas
  • Support Forum
  • Themes
  • WordPress Blog
  • WordPress Planet

Archives

  • January 2012
  • December 2011
  • November 2011
  • October 2011
  • September 2011
  • August 2011
  • July 2011
  • June 2011
  • May 2011
  • April 2011
  • March 2011
  • February 2011
  • January 2011
  • December 2010
  • November 2010
  • October 2010
  • September 2010
  • August 2010
  • July 2010
  • June 2010
  • May 2010
  • April 2010
  • March 2010
  • February 2010
  • January 2010
  • December 2009
  • November 2009
  • October 2009
  • September 2009
  • August 2009
  • July 2009
  • June 2009
  • May 2009
  • April 2009
  • March 2009
  • February 2009
  • January 2009

Company Blogs

  • Directi
  • .pw Corp Blog
  • CEOs Blog

Careers@Directi


  • About CodeChef
  • About Directi
  • CEO's Corner
  • CodeChef Campus Chapters
  • Blogger Community Program
  • User Group Outreach Program

© 2009, Directi Group. All Rights Reserved.

Sponsors