Speaker
Furano Fabrizio
(CERN IT/SDC)
Description
The Dynamic Federations project ("dynafed") enables the deployment of
scalable, distributed storage systems composed of independent storage
endpoints. While the Uniform Generic Redirector at the heart of the
project is protocol agnostic, we have focussed our effort on HTTP-based
protocols, including S3 and WebDAV. The system has been deployed on
testbeds covering the majority of the ATLAS and LHCb data, and supports
geography-aware replica selection.
The work done exploits the federation potential of HTTP to build
systems that offer uniform, scalable, catalogue-less access to
the storage and metadata ensemble and the possibility of seamless
integration of other compatible resources such as those from
cloud providers.
Dynafed can exploit the potential of the S3 delegation scheme,
effectively federating on the fly any number of S3 buckets from
different providers and applying a uniform authorization to them. This
feature has been used to deploy in production the BOINC
Data Bridge, which uses the Uniform Generic Redirector with S3 buckets
to harmonize the BOINC authorization scheme with the
Grid/X509.
We believe that the features of a loosely coupled federation of
open-protocol-based storage elements open many possibilities of
smoothly evolving the current computing models and of supporting new
scientific computing projects that rely on massive
distribution of data and that would appreciate systems that can more
easily be interfaced with commercial providers and can work
natively with Web browsers and clients.
Primary author
Furano Fabrizio
(CERN IT/SDC)
Co-authors
Laurence Field
(CERN IT/SDC)
Oliver Keeble
(CERN IT/SDC)
