SC22
November 13 – 18, 2022 (All Day)
- Kay Bailey Hutchison Convention Center in Dallas
Whether you are new to Globus or an experienced user, we would like to engage with you at SC22. Let us introduce you to all the benefits of the Globus service and update you on some of the new developments.
Visit Globus in Booth # 705
- Stop by the booth and learn what's new
- Come by the booth and say hello
- Book an in-person meeting in advance
- Ask about our new premium connectors
- See a demo
- Engage with our team
- Ask about our show specials
- Learn about our free 90-day trial subscription
Attend One of our Sessions
- Session: 12th International Workshop on Runtime and Operating Systems for Supercomputers (ROSS)
- Workshop: ROSS- Closing Invited Talk: Federated Function-as-a-Service to Power Distributed Computing Pipelines
- Presenter: Ian Foster
- Date/Time: November 13, 2022, 4:20 p.m. - 4:55 p.m. CST
- Location: D221
- Description: I report on experiences developing and deploying the funcX distributed function as a service (FaaS) platform and in employing this platform to support distributed computing pipelines that link instruments, computers (e.g., for analysis, simulation, AI model training), edge computing (e.g., for analysis), data stores, metadata catalogs, and high-speed networks. Both funcX and the Globus Flows system used to implement these pipelines combine cloud-hosted management, for reliability, with edge-hosted execution, for flexible and scalable execution. I discuss, in particular, the funcX and Globus Flows architectures; the container management strategies used in funcX to execute functions with high performance and efficiency on diverse funcX endpoints; and funcX’s integration with an in-memory data store and Globus for managing data that spans endpoints.
- Session: The 17th Workshop on Workflows in support of Large-Scale Science (WORKS22)
- Workshop: RADICAL-Pilot and Parsl: Executing Heterogenerous Workflows on HPC Platforms
- Authors/Presenters: Aymen Alsaadi, Logan Ward, Andre Merzky, Kyle Chard, Ian Foster, Shantenu Jha, Matteo Turilli
- Date/Time: November 14, 2022, 9 a.m. - 9:12 a.m.
- Location: D222
- Description: Workflow applications are becoming increasingly important to support scientific discovery. That is leading to a proliferation of workflow management systems and, thus, to a fragmented software ecosystem. Integration among existing workflow tools is a way to improve development efficiency and, ultimately, support the sustainability of the scientific workflow community. We describe our experience with integrating RADICAL-Pilot (RP) and Parsl as a way to enable users to develop and execute workflow applications with heterogeneous tasks on heterogeneous high-performance computing resources. We describe our approach to the integration of the two systems and detail the development of RPEX, a Parsl executor which uses RP as its workload manager. We develop an RP executor that enables executing heterogeneous MPI Python functions on CPUs and GPUs, and we measure the weak and strong scaling of RPEX, RP, and Parsl when providing new capabilities to two paradigmatic use cases: Colmena and Ice Wedge Polygons.
- Session: SC22 SuperCompCloud: 6th International Workshop on Interoperability of Supercomputing and Cloud Technologies
- Workshop: Assessing the Current State of AWS Spot Market Forecastability
- Authors/Presenters: Simon Caton, Matt Baughman, Christian Haas, Ryan Chard, Ian Foster, Kyle Chard
- Date/Time: November 14 2022, 11 a.m. - 11:25 CST
- Location: C141
- Description: Since 2009, Amazon has offered its unused compute capacity as AWS Spot Instances. For the first eight years of spot, pure market dynamics and high pricing variability created an ideal environment for time-series prediction. Following a pricing-scheme change in 2017, this extreme variability was removed as pricing is artificially smoothed for the end-user, therefore making it easier to accurately predict price. Nevertheless, the literature demonstrates ongoing efforts to accurately predict spot prices. To show prediction in the modern spot market is unnecessary, we train over 1.5 million ARIMA models on new and old data to demonstrate an order of magnitude improvement in accuracy for models trained on new data. Further, we show this new ease of price prediction makes spot instances ideal for large-scale, cost-aware cloud computing, as cost estimation is now trivial. Accordingly, we demonstrate that even naive prediction approaches waste less than $360 (2%) for 1,000,000 core hours.
- BoF: Workflows Community: An Update on the Community Roadmap for HPC and AI Scientific Workflows Research and Development
- Session Leader: Rafael Ferreira da Silva
- Additional Session Leaders: Rosa Badia, Kyle Chard, Olivier Terzo
- Date/Time: November 15, 2022, 12:15 - 1:15 CST
- Location: D221
- Description: The interplay of workflow technologies and HPC has been challenged by the fast rise of AI and ML technologies. Workflows empowered with ML techniques largely differ from traditional workflows running on HPC machines. In this BoF, we will bring together researchers from the workflows (https://workflows.community), HPC, and AI/ML communities that work on scientific research questions that require large-scale, distributed, and AI-heavy computing. The session will present an update on challenges, opportunities, new research directions, and future pathways, and will seek input for updating a community roadmap on HPC and AI workflows research and development.
- Featured Talk: Automating Beamline Science at Scale with Globus
- Speaker: Lee Liming
- Dates/Times: November 15, 2022, 2 p.m. and November 16, 2022, 11 a.m.
- Location: DOE Booth #1600
- Description: Globus is widely used across the DOE community for reliable file transfer and data sharing, but a growing number of computationally intensive research activities require commensurate large-scale data management. The research conducted on beamlines at the Advanced Photon Source is an exemplar. With monthly data volumes approaching the peta scale, there is a growing need for automating data flows to increase throughput and researcher productivity, and ensure the instrument remains highly utilized. We will describe how the Globus platform facilitates automation of data management tasks throughout the research process--from capture to publication--and demonstrate how such automated flows can be easily constructed and run at scale with little to no human intervention.
- BoF: Research Storage Strategies Revisited: Cloud, Capacity, Cost...and Other Things Keeping You Up at Night
- Session Leaders: Vas Vasiliadis, Rachana Ananthakrishnan
- Date/Time: November 15, 2022, 5:15 - 6:45 p.m. CST
- Location: D168
- Description: At SC21, over 50 attendees participated in robust discussion of strategies for managing storage in advanced computing environments. At the urging of many, we plan to continue the conversation this year, focusing on two themes: (1) progress made on creating unified storage environments for research computing and (2) strategies and tactics for dealing with the end of “unlimited free" cloud storage. In particular, policy and pricing model changes in cloud storage offerings have placed substantial pressure on RCD organizations to migrate to alternative storage solutions—a task that can be daunting, given the scale and diversity of data involved.
- Session: ACM and IEEE-CS Award Presentations
- Title: 2022 ACM/IEE-CS Ken Kennedy Award
- Presenter: Ian Foster
- Date/Time: November 16, 2022, 8:30-9 a.m.
- Location: Dallas Ballroom/Omni Hotel
Check back for more updates!