Managing Instrument Data with Globus
Globus provides multiple solutions for automating management of data generated by instruments, such as cryoelectron microscopes, nanopore sequencers and synchrotron beamlines.
Timers
Recurring Globus transfers can be submitted automatically on a schedule by creating a timer task. Think of it as “Globus cron”, a robust way to handle common use cases such as backing up raw instrument data for future processing and synchronization of datasets among multiple storage systems to support execution of parallel analysis pipelines.
Flows
Globus flows enable reliable execution of a sequence of tasks spanning data management and computation—as well as human input, where necessary. The common instrument flow illustrated below is representative of the types of automation that Globus provides.
Flows can be run manually but, more importantly, they can be triggered automatically based on events emitted by the instrument, such as creation of a specified file at the end of an experiment run. This allows the instrument to run uninterrupted for extended periods, while ensuring that data are reliably moved, processed, and made available to collaborators.
Getting Started with Instrument Data Automation
- Explore examples of flows and corresponding triggers.
- Browse deployed flows and manage running flows using the Globus web app.
- Experiment with building flows using a Jupyter notebook.
- Easily create and manage complex flows using the developer tools in Gladier.
Related Content
COSMIC2 Cryo-EM Project
University of California - San Diego, Science Gateways Community Institute (SGCI)
Used Globus Auth to manage user identities and access to resources, so scientists can use existing login (XSEDE, their university,...
ATLAS Experiment with Cori and Theta
National Energy Research Scientific Computing Center (NERSC), Argonne Leadership Computing Facility (ALCF)
The ATLAS experiment is a general purpose particle detector based at CERN's Large Hadron Collider (LHC), the world's largest and...