The 6th International Conference "Distributed Computing and Grid-technologies in Science and Education"

Applications of on-demand virtual clusters to high performance computing

30 Jun 2014, 16:50

20m

Conference Hall (LIT JINR)

sectional reports Section 1 - Technologies, architectures, models, methods and experiences of building distributed computing systems. Consolidation and integration of distributed resources Section 1 - Technologies, architectures, models, methods and experiences of building distributed computing systems. Consolidation and integration of distributed resources

Speakers

Mr Ivan Gankevich (Saint Petersburg State University)Mr Serob Balyan (Saint Petersburg State University)

Description

Virtual machines are usually associated with an ability to create them on demand by calling web services, then these machines are used to deliver resident services to their clients; however, pro­viding clients with an ability to run an arbitrary programme on the newly created machines is be­yond their power. Such kind of usage is useful in a high performance computing environment where most of the resources are consumed by batch programmes and not by daemons or services. In this case a cluster of virtual machines is created on demand to run a distributed or parallel programme and to save its output to a network attached storage. Upon completion this cluster is destroyed and re­sources are released. With certain modifications this approach can be extended to interactively de­liver computational resources to the user thus providing virtual desktop as a service. Experiments show that the process of creating virtual clusters on demand can be made efficient in both cases. Earlier work on this topic was presented by the authors in [1]. The concept of virtual private supercomputer, which goes back to the definition of metacomputer [2], was proposed and evaluated using different virtualization technologies and applications. Now we present a revised approach and implementation concentrating on creation of virtual machines on demand and executing distributed applications transparently in this environment. This work also develops ideas of using virtualization for creating a virtual workspace to access resources at supercomputer centers [3]. The key idea of a virtual supercomputer is to harness all available HPC resources and provide user with convenient access to them. Such a challenge can be effectively solved only using contemporary virtualization technologies. Usage of cloud technologies as well as process migration techniques can improve overall throughput of a distributed system and adapt it to problems being solved. In that way virtual supercomputer can help people efficiently run applications and focus on domain-specific problems rather than on underlying computer architecture and placement of parallel tasks. Moreover, described approach can be beneficial in utilizing stream processors and GPU accelerators dynamically assigning them to virtual machines. [1] I. Gankevich, V. Gaiduchok, D. Gushchanskiy, Y. Tipikin, V. Korkhov, A. Degtyarev, A. Bogdanov, V. Zolotarev. Virtual private supercomputer: Design and evaluation. Computer Science and Information Technologies (CSIT), 2013, pp.1-6, 23-27 Sept. 2013 doi: 10.1109/CSITechnol.2013.6710358 [2] L. Smarr and C.E. Catlett. Metacomputing. Communications of the ACM, vol. 35, no. 6, pp. 44-52, June 1992. [3] A.V. Bogdanov, A.B. Degtyarev, I.G. Gankevich, V.Yu. Gayduchok, and V. I. Zolotarev. Virtual workspace as a basis of supercomputer center. Proceedings of the 5th International Conference Distributed Computing and Grid-Technologies in Science and Education, Dubna, Russia, 2012, pp. 60-66.

Presentation materials

Slides

06_gankevich.pdf