The SIO2 project
  1. The SIO2 project
  2. SIO-2073

Set environ['required_ram'] for various worker jobs

    Details

      Description

      Part of SIO-1928.

      Jobs should report the predicted amount of RAM that they need. This should probably be a new environ parameter ("required_ram" is proposed), but another part of this issue is investigating whether there is already some similar parameter, and if there is, whether it can be used for SIO-1928.

      The list of all jobs can be found in `setup.py` of sioworkers package (`sio.jobs` list).

        Activity

        Hide
        Gerrit Gerrit added a comment -
        Change sioworkers~master~I625f7e7303bd5f69a96bde1443f8d9c38ef2ef03, patchset 14
        https://gerrit.sio2project.mimuw.edu.pl/2960

        SIO-2071 Adapt sioworkersd to respect RAM limits.

        This commit is a large part of the implementation of
        SIO-1928. It modifies sioworkersd (mainly the default
        PrioritizingScheduler) to recognize workers' total
        RAM amounts and tasks' required RAM amount. The
        core scheduling algorithm is kept the same, but
        modified to account for RAM limits, which may have
        some impact on the performance.

        For these changes to be fully functional, it's also
        necessary to implement SIO-2070 and SIO-2073.

        Change-Id: I625f7e7303bd5f69a96bde1443f8d9c38ef2ef03
        Show
        Gerrit Gerrit added a comment - Change sioworkers~master~I625f7e7303bd5f69a96bde1443f8d9c38ef2ef03, patchset 14 https://gerrit.sio2project.mimuw.edu.pl/2960 SIO-2071 Adapt sioworkersd to respect RAM limits. This commit is a large part of the implementation of SIO-1928 . It modifies sioworkersd (mainly the default PrioritizingScheduler) to recognize workers' total RAM amounts and tasks' required RAM amount. The core scheduling algorithm is kept the same, but modified to account for RAM limits, which may have some impact on the performance. For these changes to be fully functional, it's also necessary to implement SIO-2070 and SIO-2073 . Change-Id: I625f7e7303bd5f69a96bde1443f8d9c38ef2ef03
        Hide
        Gerrit Gerrit added a comment -
        Change sioworkers~master~I7f795b81ee95793d89d5a7a3bf29f9bfb205ef89, patchset 8
        https://gerrit.sio2project.mimuw.edu.pl/2967

        SIO-2073 Setting ram requirements for various worker jobs in prioritizing scheduler based on job type or given limits.

        This commit is part of SIO-1928.

        Change-Id: I7f795b81ee95793d89d5a7a3bf29f9bfb205ef89
        Show
        Gerrit Gerrit added a comment - Change sioworkers~master~I7f795b81ee95793d89d5a7a3bf29f9bfb205ef89, patchset 8 https://gerrit.sio2project.mimuw.edu.pl/2967 SIO-2073 Setting ram requirements for various worker jobs in prioritizing scheduler based on job type or given limits. This commit is part of SIO-1928 . Change-Id: I7f795b81ee95793d89d5a7a3bf29f9bfb205ef89
        Hide
        Gerrit Gerrit added a comment -
        Change sioworkers~master~I7f795b81ee95793d89d5a7a3bf29f9bfb205ef89, patchset 9
        https://gerrit.sio2project.mimuw.edu.pl/2967

        SIO-2073 Setting ram requirements for various worker jobs in prioritizing scheduler based on job type or given limits.

        This commit is part of SIO-1928.

        Change-Id: I7f795b81ee95793d89d5a7a3bf29f9bfb205ef89
        Show
        Gerrit Gerrit added a comment - Change sioworkers~master~I7f795b81ee95793d89d5a7a3bf29f9bfb205ef89, patchset 9 https://gerrit.sio2project.mimuw.edu.pl/2967 SIO-2073 Setting ram requirements for various worker jobs in prioritizing scheduler based on job type or given limits. This commit is part of SIO-1928 . Change-Id: I7f795b81ee95793d89d5a7a3bf29f9bfb205ef89
        Hide
        Gerrit Gerrit added a comment -
        Change sioworkers~master~I625f7e7303bd5f69a96bde1443f8d9c38ef2ef03, patchset 15
        https://gerrit.sio2project.mimuw.edu.pl/2960

        SIO-2071 Adapt sioworkersd to respect RAM limits.

        This commit is a large part of the implementation of
        SIO-1928. It modifies sioworkersd (mainly the default
        PrioritizingScheduler) to recognize workers' total
        RAM amounts and tasks' required RAM amount. The
        core scheduling algorithm is kept the same, but
        modified to account for RAM limits, which may have
        some impact on the performance.

        For these changes to be fully functional, it's also
        necessary to implement SIO-2070 and SIO-2073.

        Change-Id: I625f7e7303bd5f69a96bde1443f8d9c38ef2ef03
        Show
        Gerrit Gerrit added a comment - Change sioworkers~master~I625f7e7303bd5f69a96bde1443f8d9c38ef2ef03, patchset 15 https://gerrit.sio2project.mimuw.edu.pl/2960 SIO-2071 Adapt sioworkersd to respect RAM limits. This commit is a large part of the implementation of SIO-1928 . It modifies sioworkersd (mainly the default PrioritizingScheduler) to recognize workers' total RAM amounts and tasks' required RAM amount. The core scheduling algorithm is kept the same, but modified to account for RAM limits, which may have some impact on the performance. For these changes to be fully functional, it's also necessary to implement SIO-2070 and SIO-2073 . Change-Id: I625f7e7303bd5f69a96bde1443f8d9c38ef2ef03
        Hide
        Gerrit Gerrit added a comment -
        Change sioworkers~master~I625f7e7303bd5f69a96bde1443f8d9c38ef2ef03, patchset 16
        https://gerrit.sio2project.mimuw.edu.pl/2960

        SIO-2071 Adapt sioworkersd to respect RAM limits.

        This commit is a large part of the implementation of
        SIO-1928. It modifies sioworkersd (mainly the default
        PrioritizingScheduler) to recognize workers' total
        RAM amounts and tasks' required RAM amount. The
        core scheduling algorithm is kept the same, but
        modified to account for RAM limits, which may have
        some impact on the performance.

        For these changes to be fully functional, it's also
        necessary to implement SIO-2070 and SIO-2073.

        Change-Id: I625f7e7303bd5f69a96bde1443f8d9c38ef2ef03
        Show
        Gerrit Gerrit added a comment - Change sioworkers~master~I625f7e7303bd5f69a96bde1443f8d9c38ef2ef03, patchset 16 https://gerrit.sio2project.mimuw.edu.pl/2960 SIO-2071 Adapt sioworkersd to respect RAM limits. This commit is a large part of the implementation of SIO-1928 . It modifies sioworkersd (mainly the default PrioritizingScheduler) to recognize workers' total RAM amounts and tasks' required RAM amount. The core scheduling algorithm is kept the same, but modified to account for RAM limits, which may have some impact on the performance. For these changes to be fully functional, it's also necessary to implement SIO-2070 and SIO-2073 . Change-Id: I625f7e7303bd5f69a96bde1443f8d9c38ef2ef03

          People

          • Assignee:
            Szymon AcedaƄski
            Reporter:
            Pavel Senchanka
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: