[SIO-1424] Nonzero exit code of output checker should trigger System Error - SIO2

Details

Type: Improvement
Status: Closed
Priority: Minor
Resolution: Fixed
Affects Version/s: Current Version
Fix Version/s: TAG 2013/14
Component/s: Evaluation Engine / Workers
Labels:
None

Description

Rationale:
1. If the output checker crashes, we *really* want to be informed asap.
2. This would give a way of reporting errors like "the contestant's output is correct and better than model output".

Unfortunately, some checkers use exit code = 1 to report wrong answer. A possible workaround is to trigger System Error iff exit code >= 2.

Activity

Ascending order - Click to sort in descending order

Tomasz Zakrzewski made changes - 2014-02-27 11:43

Field	Original Value	New Value
Assignee		Tomasz Zakrzewski [ tommalla ]

Tomasz Zakrzewski made changes - 2014-02-27 14:45

Fix Version/s

The Sixth Sprint [ 11703 ]

Tomasz Zakrzewski made changes - 2014-03-6 13:27

Status

New [ 10000 ]

Open [ 1 ]

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-6 13:48

Change I5c3bad1265aeea52d5ce10418523ff0297ecd4df, patchset 1
https://gerrit.sio2project.mimuw.edu.pl/1997

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Show

Gerrit Gerrit added a comment - 2014-03-6 13:48 Change I5c3bad1265aeea52d5ce10418523ff0297ecd4df, patchset 1 https://gerrit.sio2project.mimuw.edu.pl/1997 SIO-1424 Nonzero exit code of output checker should trigger System Error

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-8 10:08

Change Ieb43492ba4be08f6dba0c5b0c9dbfc2dcdb9b9fe, patchset 1
https://gerrit.sio2project.mimuw.edu.pl/1999

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Show

Gerrit Gerrit added a comment - 2014-03-8 10:08 Change Ieb43492ba4be08f6dba0c5b0c9dbfc2dcdb9b9fe, patchset 1 https://gerrit.sio2project.mimuw.edu.pl/1999 SIO-1424 Nonzero exit code of output checker should trigger System Error

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-8 10:55

Change I5c3bad1265aeea52d5ce10418523ff0297ecd4df, patchset 2
https://gerrit.sio2project.mimuw.edu.pl/1997

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-8 10:55 Change I5c3bad1265aeea52d5ce10418523ff0297ecd4df, patchset 2 https://gerrit.sio2project.mimuw.edu.pl/1997 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-8 11:13

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 1
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-8 11:13 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 1 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-9 18:19

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 2
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-9 18:19 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 2 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-9 22:23

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 3
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-9 22:23 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 3 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-11 19:21

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 4
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-11 19:21 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 4 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-11 20:48

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 5
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-11 20:48 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 5 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-13 9:14

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 6
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-13 9:14 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 6 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-14 16:38

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 7
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-14 16:38 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 7 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-16 10:27

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 8
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-16 10:27 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 8 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Tomasz Zakrzewski made changes - 2014-03-18 11:14

Status

Open [ 1 ]

In Progress [ 3 ]

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-20 10:34

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 9
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-20 10:34 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 9 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-20 23:05

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 10
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-20 23:05 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 10 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-24 20:13

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 11
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-24 20:13 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 11 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Hide

Permalink

Gerrit Gerrit added a comment - 2014-03-28 7:05

Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 12
https://gerrit.sio2project.mimuw.edu.pl/2000

~~SIO-1424~~ Nonzero exit code of output checker should trigger System Error

Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Show

Gerrit Gerrit added a comment - 2014-03-28 7:05 Change I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e, patchset 12 https://gerrit.sio2project.mimuw.edu.pl/2000 SIO-1424 Nonzero exit code of output checker should trigger System Error Change-Id: I6e418b42ce88c8e5a4a6a1eee9f85881b4d0076e

Tomasz Zakrzewski made changes - 2014-03-28 19:08

Status	In Progress [ 3 ]	Resolved [ 5 ]
Assignee	Tomasz Zakrzewski [ tommalla ]	Szymon Acedański [ accek ]
Resolution		Fixed [ 1 ]

Szymon Acedański made changes - 2014-09-29 21:36

Fix Version/s		TAG 2013/14 [ 12000 ]
Fix Version/s	The Sixth Sprint [ 11703 ]

Szymon Acedański made changes - 2014-09-29 21:41

Status

Resolved [ 5 ]

Closed [ 6 ]

Transition

Time In Source Status

Execution Times

Last Executer

Last Execution Date

New

Open

22d 2h 57m

Tomasz Zakrzewski

2014-03-6 13:27

Open

In Progress

11d 21h 46m

Tomasz Zakrzewski

2014-03-18 11:14

In Progress

Resolved

10d 7h 54m

Tomasz Zakrzewski

2014-03-28 19:08

Resolved

Closed

185d 1h 32m

Szymon Acedański

2014-09-29 21:41

People

Assignee:

Szymon Acedański

Reporter:

Kuba Łącki

Votes:

0 Vote for this issue

Watchers:

3 Start watching this issue

Dates

Created:

2014-02-12 10:29

Updated:

2014-09-29 21:41

Resolved:

2014-03-28 19:08

Agile

View on Board