Next Previous Contents

4. During the contest

4.1 Monitor teams

Under the Teams menu option, you can get a general impression of the status of each team: a traffic light will show either of the following:

gray

the team has not (yet) connected to the web interface at all;

red

the team has connected but not submitted anything yet;

yellow

one or more submissions have been made, but none correct;

green

the team has made at least one submission that has been judged as correct.

This is especially useful during the practice session, where it is expected that every team can make at least one correct submission. A team with any other colour than green near the end of the session might be having difficulties.

4.2 Judging Submissions

Flow of submitted solutions

The flow of an incoming submission is as follows.

  1. Team submits solution. It will either be rejected after basic checks, or accepted and stored as a submission.
  2. The first available judgehost compiles, runs and checks the submission. The outcome and outputs are stored as a judging of this submission. Note that judgehosts may be restricted to certain contests, languages and problems, so that it can be the case that a judgehost is available, but not judging an available submission.
  3. If verification is not required, the result is automatically recorded and the team can view the result and the scoreboard is updated (unless after the scoreboard freeze). A judge can optionally inspect the submission and judging and mark it verified.
  4. If verification is required, a judge inspects the judging. Only after it has been approved (marked as verified) will the result be visible outside the jury interface. This option can be enabled by setting verification_required on the configuration settings admin page.

Submission judging status codes

The interface for jury and teams shows the status of a submission with a code.

QUEUED/PENDING

submission received and awaiting a judgehost to process it *;

JUDGING

a judgehost is currently compiling/running/testing the submission *;

TOO-LATE

submission submitted after the contest ended;

CORRECT

submission correct, problem solved;

COMPILER-ERROR

the compiler gave an error while compiling the program;

TIMELIMIT

program execution time exceeded the time defined for the problem;

RUN-ERROR

a kind of problem while running the program occurred, for example segmentation fault, division by zero or exitcode unequal to 0;

NO-OUTPUT

there was no output at all from the program;

WRONG-ANSWER

the output of the program did not exactly match the expected output;

* in the team interface a submission will only show as PENDING to prevent leaking information of problem time limits. The jury can see whether a submission is QUEUED or JUDGING. In case of required verification, a submission will show as PENDING to the team until the judging has been verified.

Under the Submissions menu, you can see submitted solutions, with the newest one at the top. Click on a submission line for more details about the submission (team name, submittime etc), a list of judgings and the details for the most recent judging (runtime, outputs, diff with testdata). There is also a switch available between newest 50, only unverified, only unjudged or all submissions. The default (coloured) diff output shows differences on numbered lines side by side separated by a character indicating how the lines differ: ! for different contents, $ for different or missing end-of-line characters, and one of <> when there are no more lines at the end of the other file.

Under the submission details the `view source code' link can be clicked to inspect the source code. If the team has submitted code in the same language for this problem before, a diff output between the current and previous submission is also available there.

It is possible to edit the source code and resubmit it if you have a team associated to your user. This does not have any effect for the teams, but allows a judge to perform a `what if this was changed'-analysis.

A submission can have multiple judgings, but only one valid judging at any time. Multiple judgings occur when rejudging, see Rejudging.

Rejudging

In some situations it is necessary to rejudge one or more submissions. This means that the submission will re-enter the flow as if it had not been judged before. The submittime will be the original time, but the program will be compiled, run and tested again.

This can be useful when there was some kind of problem: a compiler that was broken and later fixed, or testdata that was incorrect and later changed. When a submission is rejudged, the old judging data is kept but marked as `invalid'.

You can rejudge a single submission by pressing the `Rejudge' button when viewing the submission details. It is also possible to rejudge all submissions for a given language, problem, team or judgehost; to do so, go to the page of the respective language, problem, team or judgehost, press the `Rejudge all' button and confirm.

There are two different ways to run a rejudging, depending on whether the create rejudging button toggled:

  1. Without this button toggled, an "old-style" rejudging is performed where the results are directly made effective.
  2. When toggled, a "rejudging" set is created, and all affected submissions are rejudged, but the new judgings are not made effective yet. Instead, the jury can inspect the results of the rejudging (under the rejudging tab). Based on that the whole rejudging, as a set, can be applied or cancelled, keeping the old judgings as is.

Submissions that have been marked as `CORRECT' will not be rejudged. Only DOMjudge admins can override this restriction using a tickbox.

Teams will not get explicit notifications of rejudgings, other than a potentially changed outcome of their submissions. It might be desirable to combine rejudging with a clarification to the team or all teams explaining what has been rejudged and why.

Ignored submissions

Finally, there is the option to ignore specific submissions using the button on the submission page. When a submission is being ignored, it is as if was never submitted: it will show strike-through in the jury's and affected team's submission list, and it is not visible on the scoreboard. This can be used to effectively delete a submission for some reason, e.g. when a team erroneously sent it for the wrong problem. The submission can also be unignored again.

4.3 Clarification Requests

Communication between teams and jury happens through Clarification Requests. Everything related to that is handled under the Clarifications menu item.

Teams can use their web interface to send a clarification request to the jury. The jury can send a response to that team specifically, or send it to all teams. The latter is done to ensure that all teams have the same information about the problem set. The jury can also send a clarification that does not correspond to a specific request. These will be termed `general clarifications'.

Under Clarifications, three lists are shown: new clarifications, answered clarifications and general clarifications. It lists the team login, the problem concerned, the time and an excerpt. Click the excerpt for details about that clarification request.

Every incoming clarification request will initially be marked as unanswered. The menu bar shows the number of unanswered requests. A request will be marked as answered when a response has been sent. Additionally it's possible to mark a clarification request as answered with the button that can be found when viewing the request. The latter can be used when the request has been dealt with in some other way, for example by sending a general message to all teams.

An answer to a clarification request is made by putting the text in the input box under the request text. The original text is quoted. You can choose to either send it to the team that requested the clarification, or to all teams. In the latter case, make sure you phrase it in such a way that the message is self-contained (e.g. by keeping the quoted text), since the other teams cannot view the original request.

The menu on every page of the jury interface will mention the number of unanswered clarification requests: ``(1 new)''. This number is automatically updated, even without reloading the page.


Next Previous Contents