The DTNs offer direct access to global NERSC file systems & Cori cscratch1. These servers include high-bandwidth network interfaces and are tuned for efficient data transfers. The Data Transfer Nodes (DTNs) are dedicated servers for moving data at NERSC. NERSC maintains managed endpoints for optimized data transfers. The Globus website contains extensive documentation. It is accessible to all NERSC users, with options for web based interaction with service, REST/API for scripted interactions (in Bash, Python and other languages) or the Globus Connect Server & Personal for setting up additional remote endpoints such as your personal laptop. It has a reliable & easy-to-use web-based service via or. Globus is the recommended tool for moving data in & out of NERSC. How do I get my data into NERSC? ¶ Globus ¶ It is not a replacement for the full NERSC documentation, rather it is a curated subset of our documentation that focuses on the needs of this user community. This page presents best practices for experimental scientists using NERSC, ranging from data management, running jobs and edge services. In addition to the standard best practices, for this community we have additional recommendations. Changes in policies are as important to address as technical challenges.Experimental facilities will require new modes of interacting with the systems including notebooks and faster queue turn-around.New strategies for resilient workflows are required.As science increasingly becomes a community effort, the need to share, transfer, search and access data becomes even more important.New approaches are needed for analyzing large datasets including advanced statistics and machine learning.Scientists require support for analysis software and tools, many of which differ significantly from traditional simulation software.
HPC is required to analyze data from experimental facilities - but changes are needed to both application workflows and HPC environments.This user community has specific needs that we have heard through requirements gathering and our long experience working with this community, including: NERSC has an increasing number of users from the DOE's experimental and observational facilities. Best Practices for Experimental Science NERSC Users ¶
0 kommentar(er)
