Speaker
Description
"The High Energy cosmic Radiation Detector (HERD) is a major international space astronomy and particle astrophysics experiment planned for installation on the Chinese Space Station (CSS) around 2027. Its primary scientific objectives include precise measurement of high-energy cosmic rays up to the PeV range, indirect detection of dark matter, and observation of high-energy gamma rays.
It is estimated that HERD experiment will generate over 70PB data from during 10 years operation, encompassing simulated, raw, reconstructed, calibration and engineering datasets. To address the challenges of data management, data processing and distribution, we designed and developed the HERD Data Management and Workflow system. This system is structured around three core functional modules to ensure streamlined operations.
The Data Management module is responsible for the metadata catalog, data distribution across infrastructure, and data services. The Data Workflow module automates the entire data processing pipeline, including data simulation, reconstruction, and transfer. The workflow is designed to minimize manual intervention, ensuring consistency and reliability from data acquisition to final data analysis. The Payload Operation Monitoring module monitors the telemetry and engineering data from the HERD payloads in real-time. It is designed to detect anomalies and trigger alerts, thereby ensuring the instrument's health and operational continuity.
This paper firstly introduces the HERD experiment and data challenges, and then details the system's architecture, elaborates on the implementation of these three modules, and reports on the current development progress. Finally, it concludes with a summary and outlines the future plan for the system's deployment and refinement."