The SIO2 project
  1. The SIO2 project
  2. SIO-1797

Check for anomalies in szkopuł/oitimetool running times

        Details

        • Type: Task Task
        • Status: Closed Closed
        • Priority: Major Major
        • Resolution: Fixed
        • Affects Version/s: Current Version
        • Fix Version/s: TAG 2015/16
        • Component/s: OIOIOI
        • Labels:

          Description

          Need to check all public problems available in problemset for anomalies in szkopuł/oitimetool running times ratio.
          Especially I expect some problems to have longer running times on szkopuł than oitimetool, but most of them should have ratio around 1:1.

          Analysis of small sample (9 solutions/problems): SIO-1754

          The goal is to identify problems which timelimits should be scaled with something else than 1 and compute better ratio for them.
          Another goal is to generate chart like in SIO-1754 .
          1. File
            oitimrunner.py
            5 kB
            Sebastian Jaszczur
          2. File
            chart.py
            4 kB
            Sebastian Jaszczur
          1. oitimetool_times2.png
            652 kB

            Issue Links

            blocks

            Bug - A problem which impairs or prevents the functions of the product. SIO-1812 Switch to oitimetool on szkopuł

            • Major - Major loss of function.
            • Closed - The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              Sebastian Jaszczur added a comment - - edited
              After analysis of most of problems (around 600) indicates that maximum of 22 problems would cause problems after scaling timelimits 1:1. That's around 3%.
              It's worth noticing that 17 problems have similar problems already (but it's not the subset of previous problems; they share 5 elements).

              From those 22 problems:
              2 were some kind of bugs and oitimetool gave time of around 20 minutes when timelimit (and original runtime) was around few seconds. I don't know why this has happened, though.
              15 of them would lose max 3 points.
              3 of them would lose 5-6 points.
              1 would lose around 14 points.
              1 would lose around 23 points.

              So generally it's not so bad and with Accek we've agreed on scaling timelimits 1:1, as it's fairly OK and it's effortless. To those few problems that may lose some points - we will create a tool for searching for them, probably available from manage.py. Ticket: SIO-1819
              Show
              Sebastian Jaszczur added a comment - - edited After analysis of most of problems (around 600) indicates that maximum of 22 problems would cause problems after scaling timelimits 1:1. That's around 3%. It's worth noticing that 17 problems have similar problems already (but it's not the subset of previous problems; they share 5 elements). From those 22 problems: 2 were some kind of bugs and oitimetool gave time of around 20 minutes when timelimit (and original runtime) was around few seconds. I don't know why this has happened, though. 15 of them would lose max 3 points. 3 of them would lose 5-6 points. 1 would lose around 14 points. 1 would lose around 23 points. So generally it's not so bad and with Accek we've agreed on scaling timelimits 1:1, as it's fairly OK and it's effortless. To those few problems that may lose some points - we will create a tool for searching for them, probably available from manage.py. Ticket: SIO-1819
              Hide
              Permalink
              Sebastian Jaszczur added a comment -
              Raw scripts used to generate results and chart.
              Show
              Sebastian Jaszczur added a comment - Raw scripts used to generate results and chart.

                People

                • Assignee:
                  Szymon Acedański
                  Reporter:
                  Sebastian Jaszczur
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  1 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: