Speaker
Mr
Mikhail Borodin
(NRNU MEPHI, NRC KI)
Description
The data processing and simulation needs of the ATLAS experiment at LHC grow continuously, as more data are collected and more use cases emerge. For data processing the ATLAS experiment adopted the data transformation approach, where software applications transform the input data into outputs. In the ATLAS production system, each data transformation is represented by a task, a collection of many jobs, dynamically submitted by the ATLAS workload management system (PanDA/JEDI) and executed on the Grid, clouds and supercomputers. Patterns in ATLAS data transformation workflows composed of many tasks provided a scalable production system framework for template definitions of the many-tasks workflows. User interface and system logic of these workflows are being implemented in the Database Engine for Tasks (DEFT). Such development required using modern computing technologies and approaches. We report technical details of this development: database implementation, server logic and Web user interface technologies.
Primary author
Mr
Mikhail Borodin
(NRNU MEPHI, NRC KI)
Co-authors
Alexandre Vaniachine
(Argonne)
Dr
Alexei Klimentov
(Brookhaven National Lab)
Mr
Dmitry Golubkov
(Big Data Laboratory, National Research Centre "Kurchatov Institute", Moscow, Russia)
Dr
Kaushik De
(Physics Department, University of Texas at Arlington, TX, United States of America)
Dr
Ruslan Mashinistov
(Big Data Laboratory, National Research Centre "Kurchatov Institute", Moscow, Russia)
Dr
Tadashi Maeno
(Physics Department, Brookhaven National Laboratory, United States of America)
