What the Graders Actually Check

An auto-graded assessment measures three things and nothing else: whether your function returns the right answer on inputs you never see, whether it finishes inside a time limit, and how many of those inputs it got through before the clock ran out. Everything you might want it to notice, that your approach was sensible, that you nearly had it, that the failing case is a one-character fix, is invisible.

That narrowness has consequences worth internalising, because they are the difference between a candidate who scores 4 out of 4 and one who scores 2 and cannot explain why.

Hidden tests are where the marks are

You are typically shown two or three sample cases and graded on twenty or thirty. The samples are illustrative. They are chosen to explain the problem, which is close to the opposite of being chosen to break your code.

The hidden set does the breaking, and it is boringly predictable in what it reaches for:

• The empty input. An empty list, an empty string, zero orders.
• The single element. Where a loop comparing i against i - 1 never runs, or runs once with a negative index.
• Everything identical. All prices equal, all timestamps equal, which turns a sort into a question about stability and a tie-break into a question about whether you defined one.
• The maximum size in the constraints. Not a large input, the large input, sized so that the intended complexity passes and the next one up does not.
• The pathological ordering. Already sorted, reverse sorted, or the arrangement that turns a hash map into a linked list.

Writing those five cases yourself before you submit costs three minutes and is the single highest-return habit in the format. You cannot see the hidden tests, but you can see the constraints, and the constraints are where the hidden tests come from.

"Suboptimal solutions fail" is a time limit

One widely reported assessment warns that "suboptimal solutions fail", and candidates read that as a stylistic preference. It is not. It is a wall-clock limit on the harness, chosen so that a solution one complexity class above the intended one does not finish.

The arithmetic is worth doing once. Suppose $n = 10^5$ and the limit is a few seconds.

At roughly $10^7$ simple operations per second in interpreted Python, and around a hundred times that in compiled C++, the second finishes comfortably in either and the first finishes in neither. There is no partial credit for being close: a quadratic solution to a problem with $n = 10^5$ scores whatever the small tests are worth and nothing else.

This is why the constraint block is the most information-dense part of an assessment problem. A bound of $n \le 20$ is asking for exponential search. $n \le 2000$ tolerates quadratic. $n \le 10^5$ means sorting, a hash map, a heap or a single pass. $n \le 10^9$ means you are not iterating over $n$ at all and the answer is arithmetic or binary search.

What the grader cannot see, and who checks it later

The auto-grader has no opinion about your variable names. The next round does. Some firms state plainly that code is read after the tests pass, longer data assessments have carried a clean-code requirement in the brief itself, and one well-known onsite is built around layering new requirements onto the solution you already wrote, which is a direct test of whether the first version can be modified at all.

So the format asks for two things that pull in opposite directions: get to correct fast, and leave behind something a person can extend. The next section is about doing both, because they conflict less than they appear to.