Speaker
Description
The SPD Online Filter is a critical intermediate system that bridges the continuous data stream from the DAQ with offline processing stages at geographically distributed computing centers. Its primary purpose is to reduce the vast amounts of raw data — by at least a factor of 20 — to enable full processing, long-term storage, and subsequent analysis. Because the data arrives without a hardware trigger, a multi-step software-based filtering approach is required to effectively filter the detector data in real time. Building on this multi-step framework, the SPD Online Filter architecture unites robust hardware with sophisticated middleware to manage complex data workflows. The DAQ aggregates detector signals into uniform time blocks, storing them in files for further processing. The middleware then coordinates a chain of processing steps: the Data & Storage Management System oversees the entire data lifecycle—handling registration, cataloging, consistency checks, cleanup, and storage; the Workflow Management System orchestrates the processing chain by defining tasks on a predefined graph of operations; and the Workload Management System, working in tandem with a Pilot Agent, creates and supervises jobs on compute nodes, ensuring seamless execution and secondary data generation. This report details the coordinated interactions among these components, demonstrating how they collectively enable robust, scalable, and real-time data processing to meet the requirements of the SPD experiment.
