Moving to petascale with the research community
November 15, 2021 | ESnet news
The Energy Sciences Network (ESnet) has long been focused on developing ways to streamline workflows and reduce network operational burdens on the scientific programs and researchers they serve directly, as well as those on behalf of the entire R&E network community. Building on the successful Science DMZ design pattern and the Petascale DTN project, the Data Mobility Exhibition (DME) was developed to improve the predictability of data movement between research sites and universities. Many sites use perfSONAR to test end-to-end network performance. The DME allows sites to take this a step farther and test end to end data transfer performance.
Past Petascale Data Transfer Node (DTN) data mobility efforts have helped large scientific data transfers. For example, 768 TB of DESI data were sent via ESnet between OLCF and NERSC automatically via Globus over a 20 hour period. Despite the interruption from maintenance activity at ORNL, the transfer was seamlessly reconnected without any user involvement.
To date, over 100 DTN operators have used DME benchmarking resources to tune their own data transfer performance. In addition, the DME has been added to the NSF-funded Engagement and Performance Operations Center (EPOC) program’s six main scientific networking consulting support services, bringing this capability to a wide set of US Research Universities.