Managing Instrument Data with Globus
Globus provides multiple solutions for automating management of data generated by instruments, such as cryoelectron microscopes, nanopore sequencers and synchrotron beamlines.
Recurring Globus transfers can be submitted automatically on a schedule by creating a timer task. Think of it as “Globus cron”, a robust way to handle common use cases such as backing up raw instrument data for future processing and synchronization of datasets among multiple storage systems to support execution of parallel analysis pipelines.
Globus flows enable reliable execution of a sequence of tasks spanning data management and computation—as well as human input, where necessary. The common instrument flow illustrated below is representative of the types of automation that Globus provides.
Flows can be run manually but, more importantly, they can be triggered automatically based on events emitted by the instrument, such as creation of a specified file at the end of an experiment run. This allows the instrument to run uninterrupted for extended periods, while ensuring that data are reliably moved, processed, and made available to collaborators.
Getting Started with Instrument Data Automation
University of California - San Diego, Science Gateways Community Institute (SGCI)
Used Globus Auth to manage user identities and access to resources, so scientists can use existing login (XSEDE, their university,...
National Energy Research Scientific Computing Center (NERSC), Argonne Leadership Computing Facility (ALCF)
The ATLAS experiment is a general purpose particle detector based at CERN's Large Hadron Collider (LHC), the world's largest and...