Speaker
Dr
Stanislav Polyakov
(SINP MSU)
Description
Modern supercomputers employ schedulers to distribute the queued jobs between the computational nodes and maximize their load. The jobs can vary in size (number of required computational units, planned time) and it is impossible to schedule them with 100% efficiency. Additionally, some jobs are completed ahead of time which can also decrease the load. In practice the average load of supercomputers seldom exceeds 90% and often is below 80%.
Theoretically, any slots left empty by the scheduler can be used by jobs that require a single scheduling CPU slot and sufficiently short execution time. In practice, there are numerous computational problems that do not require parallelization and therefore need a single CPU slot but have significant time requirements. A queue of jobs solving these problems can be used to fill short schedule slots if the jobs are migrated to different nodes or saved and returned to the queue before the end of the allotted time. It is possible with the use of container virtualization that allows to start, stop, and migrate containers relatively quickly.
We develop a simulation model of a system intended to increase the load of supercomputers using an additional queue of non-parallel jobs packed in containers. Here we present an estimation of the load of supercomputers using different scheduler algorithms. The estimation is based on simulation with a simulated queue consisting of jobs with randomly generated parameters (number of CPUs, planned time, actual completion time). The chosen distribution of the parameters is based on typical data of supercomputers. We also present an estimation of the expected load increase resulting from the use of container migration.
Primary author
Dr
Stanislav Polyakov
(SINP MSU)
Co-authors
Elena Petrova
(Skobeltsyn Institute of Nuclear Physics, Lomonosov Moscow State University)
Mr
Minh Duc Nguyen
(Skobeltsyn Institute of Nuclear Physics, Lomonosov Moscow State University)
