[SIO-1424] Nonzero exit code of output checker should trigger System Error - SIO2

Details

Type: Improvement
Status: Closed
Priority: Minor
Resolution: Fixed
Affects Version/s: Current Version
Fix Version/s: TAG 2013/14
Component/s: Evaluation Engine / Workers
Labels:
None

Description

Rationale:
1. If the output checker crashes, we *really* want to be informed asap.
2. This would give a way of reporting errors like "the contestant's output is correct and better than model output".

Unfortunately, some checkers use exit code = 1 to report wrong answer. A possible workaround is to trigger System Error iff exit code >= 2.

Activity

Ascending order - Click to sort in descending order

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-6 13:48

Change I5c3bad1265aeea52d5ce10418523ff0297ecd4df, patchset 1
https://gerrit.sio2project.mimuw.edu.pl/1997

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Show

Gerrit Gerrit added a comment - 2014-03-6 13:48 Change I5c3bad1265aeea52d5ce10418523ff0297ecd4df, patchset 1 https://gerrit.sio2project.mimuw.edu.pl/1997 SIO-1424 Nonzero exit code of output checker should trigger System Error

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-8 10:08

Change Ieb43492ba4be08f6dba0c5b0c9dbfc2dcdb9b9fe, patchset 1
https://gerrit.sio2project.mimuw.edu.pl/1999

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Show

Gerrit Gerrit added a comment - 2014-03-8 10:08 Change Ieb43492ba4be08f6dba0c5b0c9dbfc2dcdb9b9fe, patchset 1 https://gerrit.sio2project.mimuw.edu.pl/1999 SIO-1424 Nonzero exit code of output checker should trigger System Error

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-8 10:55

Change I5c3bad1265aeea52d5ce10418523ff0297ecd4df, patchset 2
https://gerrit.sio2project.mimuw.edu.pl/1997

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-8 10:55 Change I5c3bad1265aeea52d5ce10418523ff0297ecd4df, patchset 2 https://gerrit.sio2project.mimuw.edu.pl/1997 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-8 11:13

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 1
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-8 11:13 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 1 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-9 18:19

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 2
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-9 18:19 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 2 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-9 22:23

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 3
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-9 22:23 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 3 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-11 19:21

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 4
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-11 19:21 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 4 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-11 20:48

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 5
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-11 20:48 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 5 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-13 9:14

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 6
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-13 9:14 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 6 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-14 16:38

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 7
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-14 16:38 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 7 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-16 10:27

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 8
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-16 10:27 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 8 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-20 10:34

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 9
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-20 10:34 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 9 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-20 23:05

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 10
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-20 23:05 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 10 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-24 20:13

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 11
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-24 20:13 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 11 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-28 7:05

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 12
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-28 7:05 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 12 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

People

Assignee:

Szymon Acedański

Reporter:

Kuba Łącki

Votes:

0 Vote for this issue

Watchers:

3 Start watching this issue

Dates

Created:

2014-02-12 10:29

Updated:

2014-09-29 21:41

Resolved:

2014-03-28 19:08

Agile

View on Board