Updated 15/02/2024
DTE Infrastructure Component

FTS3

Federated Data Infrastructure
Image

Description

The software underlying a service responsible for globally distributing the majority of the LHC data across the WLCG infrastructure.

FTS is the software underlying a service responsible for globally distributing the majority of the LHC data across the WLCG infrastructure, in addition to other scientific communities. It is a low-level data movement service, responsible for reliable bulk transfer of files from one site to another while allowing participating sites to control the network resource usage.

Transferring large volumes of data between facilities requires a component that manages those transfers: monitoring their progress, cancelling transfers that have stalled or that take too long, and retrying failed transfers (where appropriate).

Deploying FTS provides a common service for handling such transfers, allowing higher-level data management; e.g., bandwidth shaping links between facilities.

FTS is the engine that transfers data within the Data Lake, when a rule is added to Rucio that the current data replicas do not satisfy.

Target Audience
+

FTS is a core component that most users do not access directly.  Instead, FTS is used by any DT user or manager who is (directly or otherwise) managing data locality.

FTS is used when a DT user or manager needs some data to be located at some facility and that data is not currently available.

License
+

Apache 2.0

Created by
+

Release Notes

FTS is a fully established project, independent of the interTwin project.  The software is production-ready, at TRL 9, and hardened with many years of production-critical use.

Future Plans

Federated data management within interTwin’s DTE blueprint was developed based on using FTS to transfer data between sites.  The FTS project has a well-established support process that is science-agnostic and community-driven.  Deployments of the interTwin DTE blueprint that take advantage of FTS will be supported through the FTS project.

As the interTwin project sees the initial adoption of FTS by new communities, some additional features have been identified as useful.  Not all of these have been implemented within the interTwin project.  Those that have not have been documented and made available as possible topics for future projects.