The SIO2 project
  1. The SIO2 project
  2. SIO-2405

Prioritize certain kinds of statuses when aggregating ACM-style test results

        Details

        • Type: Improvement Improvement
        • Status: New New
        • Priority: Major Major
        • Resolution: Unresolved
        • Affects Version/s: Current Version
        • Fix Version/s: None
        • Component/s: Evaluation Engine / Workers
        • Labels:

          Description

          In ACM-style contests, the user-visible result for a submit is "AC" when all tests pass, and some type of error status when one or more tests fail.

          Currently, if multiple tests fail with different statuses, the "first" failure is shown.
          Moreover, "first" is defined according to some internal order, different from the order visible in UI.

          It'd be preferable if the status to display was chosen based on type of failure, and not order of tests.
          For example, RE could take precedence over WA, then MLE, then TLE, etc.

            Activity

            There are no comments yet on this issue.

              People

              • Assignee:
                Alicja Kluczek
                Reporter:
                Wojciech Dubiel
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: