The new execution Time for your submission

1 min read

To start with, let us revisit our definition of Time Limit: It is the time in which your program must read, process, and output the result of an input test file. It is defined in detail here.

Up until now, for each submission we used to display the total execution time across all the test files. What this means is that if a problem had multiple input files, then the reported execution time of your submission used to be the sum of times taken for execution on each test file. Thus, many a times, the reported time limit would be higher than the Time Limit specified for the problem. Most of you have gotten used to this, but for every new user this has created a lot of confusion.

Example: If the problem has 3 test files a, b, and c and the time limit for the problem is T. The time taken to pass each of them is ta, tb, and tc respectively. Then currently time displayed is (ta + tb + tc). And may a times (ta+tb+tc) > T, causing the confusion.

Additionally, sometimes revealing the cumulative time of execution reveals a lot more about the test cases than what the problem authors want to reveal during a running contest. This applies more to the Challenge problem where users make many solutions to try and extract information about the test data.

And hence, from the feedback that we have received from our community and problem authors, we have changed the way we will report time against your submission. From now on, the time displayed for your submission will be the maximum of the times taken by your solution to execute each test file. Let us explain with another example.

Example: If the problem has 3 test files a, b, and c and the time limit for the problem is T. The time taken to pass each of them is ta, tb, and tc respectively, with tb being the maximum time taken among the 3 test files. So, from now, the time displayed will be tb, making the time displayed <= T.

This ensures that the reported time will always be less than or equal to the Time Limit of the specified problem which is what is expected by everyone.

So, next time you are looking at that Time column against a submission, do keep this new change in mind. However, we will not be making this change in the previously closed contests. This is only applicable to our new contests and the problems in the practice section.

Do let us know what you think of this new change we have brought in for you at feedback@codechef.com

Till next time. Adios.

Regards,
Rudreshwar,
Team CodeChef

Going For Gold: Meet The IOI 2020 Singapore Finalists…

The 32nd International Olympiad in Informatics is upon us, and now we know the names of the young, Indian coders who made it to...
riddhi_225
2 min read

A Learning Program That You Always Wanted!

tl;dr: CodeChef is launching live courses on Competitive Programming, taught by the most passionate competitive programmers, and helped by a great team of teaching...
anup
3 min read

We Expanded Our Kitchen With A New YouTube Channel!

Since the beginning of time, we have aimed to provide various learning opportunities for the community. For the last 11 years, we have done...
debanjan321
2 min read

2 Replies to “The new execution Time for your submission”

  1. I think this is a good change, but I think this hasn’t been implemented for challenge problems? I submitted a solution for a challenge problem for Practice (the one for the most recent long contest) and the reported time seemed to be the sum over all test cases (because it was much larger than the individual time limit per test case). I guess this is due to challenge problems and the “standard” problems using a different master judge? Anyway, what I am trying to say is that it would be good to provide complete information – so if this change won’t apply for challenge problems, let us know.

    1. You are right. This change will be done for all problems. However because the problem author was not updated on this change, he did not use the new master judge. If you notice, the same also happened only for the challenge problem during the contest. We will change the judge of the Challenge problem in the practice section and rejudge those solutions. Thanks for bringing this to our notice.

Leave a Reply