30 September 2019 to 4 October 2019
Montenegro, Budva, Becici
Europe/Podgorica timezone

MC2E: Meta-Cloud Computing Environment for HPC

30 Sept 2019, 17:00
30m
Splendid Conference & SPA Resort, Conference Hall Petroviċa

Splendid Conference & SPA Resort, Conference Hall Petroviċa

Plenary Plenary

Speaker

Ruslan Smeliansky (Lomonosov Moscow State University)

Description

Modern practical research in physics, chemistry and biology has shifted in the area of simulations, experimental results’ processing and data mining, thus imposing immense demands on computational resources. The problem is that due to the heterogeneous nature such resources may have a high variance in their load. So users may wait for weeks until their job is done, even though there is plenty of resources available on other platforms. Such problem arises because various platforms may have significantly different APIs and when researchers used to work with one interface it’s often expensive to fit their software to work with some other interface. In this research we present MC2E - an environment for academic multidisciplinary research. MC2E aggregates heterogeneous resources such as private/public clouds, HPC clusters and supercomputers under a unified easy-to-use interface.

Summary

Today's research in various fields such as physics, chemistry and biology have shows large demands in computational resources due to the complexity of tasks performed. Such resources are often provided as supercomputers and clusters for High Performance Computing (HPC).

The problem is that such platforms may have a high variance in their load. So more popular platforms may have large queues and significant waiting times, while other platforms are vacant.

On-demand clouds could help solving this problem by offering virtualized resources customized for specific purposes. Such clouds offer more flexible and convenient platform for researchers to execute their computational tasks, but their heterogeneity makes it hard to switch between platforms if some platform becomes highly loaded or inaccessible.

This happens because various resource providers have different interfaces (APIs) for task submission. So in order to change the target platform researchers need to spend time and resources to adjust their software for the new API.

In this research we propose MC2E - an environment for multidisciplinary academic research that aggregates heterogeneous resources such as private/public clouds, HPC clusters and supercomputers under a unified easy-to-use interface.

Comparing with “traditional” resource orchestration in data centers, that use free tools like OpenStack or commercial provided by VMware, MC2E offers a number of new features/opportunities and advantages:

  • MC2E provides higher level of resource control (a set of platforms instead of a single local data center or HPC cluster);

  • It provides users with more flexible capabilities to define virtual environments, more types of resources and services;

  • It supplies higher quality of resource scheduling and utilization;

  • It relieves a user from tedious system administration tasks;

  • It specifies a unified way to describe and support the data center (or HPC cluster) service livecycle. Allowing an MC2E user to use his software for providing experiments on MC2E infrastructure.

MC2E enlarges the concepts of PaaS and IaaS to the scientific applications area. We believe that it could be of great help to research groups that work jointly and need a shared virtual collaboration environment with resources from different geographically distributed platforms.

Primary authors

Ivan Petrov (Lomonosov Moscow State University) Ruslan Smeliansky (Lomonosov Moscow State University) Vitaly Antonenko (Lomonosov Moscow State University)

Co-authors

Anatoly Bahmurov (Lomonosov Moscow State University) Donggang Cao (Peking University) Min Chen (Huazhong University of Science & Technology) Xiangqun Chen (Peking University) Zhenchun Huang (Tsinghua University)

Presentation materials