Details
-
Type: Improvement
-
Status: Closed
-
Priority: Major
-
Resolution: Fixed
-
Affects Version/s: Current Version
-
Fix Version/s: TAG 2017/18 Sprint 1
-
Component/s: Evaluation Engine / Workers
-
Labels:None
Description
Part of SIO-1928.
Right now, the worker config defines WORKER_CONCURRENCY parameter which is calculated from both CPU number and (RAM / average RAM for a job). This parameter should be split into WORKER_CONCURRENCY that only depends on CPU count, and WORKER_RAM that declares the amount of RAM that can be used.
The default value should leave some RAM for OS usage.
Right now, the worker config defines WORKER_CONCURRENCY parameter which is calculated from both CPU number and (RAM / average RAM for a job). This parameter should be split into WORKER_CONCURRENCY that only depends on CPU count, and WORKER_RAM that declares the amount of RAM that can be used.
The default value should leave some RAM for OS usage.
Activity
Pavel Senchanka
made changes -
Field | Original Value | New Value |
---|---|---|
Fix Version/s | TAG 2017/18 Sprint 1 [ 12800 ] |
Norbert Siwek
made changes -
Assignee | Norbert Siwek [ norbi_s ] |
Norbert Siwek
made changes -
TAG Developer | Norbert Siwek [ norbi_s ] |
Norbert Siwek
made changes -
Status | New [ 10000 ] | Open [ 1 ] |
Norbert Siwek
made changes -
Status | Open [ 1 ] | Resolved [ 5 ] |
Assignee | Norbert Siwek [ norbi_s ] | Szymon Acedański [ accek ] |
Resolution | Fixed [ 1 ] |
Szymon Acedański
made changes -
Status | Resolved [ 5 ] | Closed [ 6 ] |
Transition | Time In Source Status | Execution Times | Last Executer | Last Execution Date | |||||
---|---|---|---|---|---|---|---|---|---|
|
6d 4h 10m | 1 | Norbert Siwek | 2018-03-19 16:34 | |||||
|
29d 4h 26m | 1 | Norbert Siwek | 2018-04-17 22:01 | |||||
|
342d 19h 39m | 1 | Szymon Acedański | 2019-03-26 16:40 |
https://gerrit.sio2project.mimuw.edu.pl/2960
SIO-2071Adapt sioworkersd to respect RAM limits.This commit is a large part of the implementation of
SIO-1928. It modifies sioworkersd (mainly the defaultPrioritizingScheduler) to recognize workers' total
RAM amounts and tasks' required RAM amount. The
core scheduling algorithm is kept the same, but
modified to account for RAM limits, which may have
some impact on the performance.
For these changes to be fully functional, it's also
necessary to implement
SIO-2070andSIO-2073.Change-Id: I625f7e7303bd5f69a96bde1443f8d9c38ef2ef03