CADES Data Transfer Nodes (DTNs) allow for efficient movement of large data sets into and out of ORNL’s network. There are several transfer tools and protocol options to fit your needs.
- Globus provides a web interface or command line tools to transfer data between institutional endpoints, your own personal endpoints, or securely share access to your data. This is the preferred method of transfer for CADES. See below for a Globus tutorial.
- Secure copy (scp) via the command line to and from storage locations, including ORNL workstations.
- scp [email protected]:/path/to/directory/example.txt [email protected]:/path/to/directory/
- Secure (or SSH) file transfer protocol (SFTP) can be used to transfer files between two remote storage locations (similar to scp) but also allows the user to list directories and see content. You can use SFTP as long as you have SSH access to that host.
- sftp username@remote_hostname_or_IP
- Graphical clients (SFTP and S3) will allow you to use a graphical user interface with drag-and-drop capabilities. CADES maintains documentation for CyberDuck and WinSCP.
Globus File Transfer Tutorial
Getting Started and Signing In
Globus is primarily used via its web interface, though it is possible to download a personal client or use command line tools.
- Navigate to the Globus website (https://www.globus.org/) and click Log in.
- Select your organization Oak Ridge National Laboratories from the drop-down menu and select Continue.
- Use your UCAMS username and password to log in.
- If you have an existing Globus account, you may choose to link them at this time, or skip to the next step, by clicking No thanks, continue.
- Accept the user agreement and Continue. The next screen will ask you to Allow permissions.
Activating Endpoints
- Click in the Endpoint box on the left side and search for CADES-OR (CADES Open Research).
- You will be redirected to enter your UCAMS credentials.
- Authenticating the Endpoint with your credentials is known as Endpoint Activation and can be done when adding and using an Endpoint for the first time, or can be completed by navigating to the “Manage Endpoints” screen (Endpoints → Endpoint List → activate).
- Once the endpoint is set you can modify the path to point to your file/data. In this example, we will connect to Lustre storage: /lustre/or-scratch/cades-birthright/$USER
- On the right side, set the endpoint. We will use OLCF Titan’s file system. Search for OLCF ATLAS and complete the Endpoint Activation steps.
Globus File Transfers
- Find the endpoints (on the left and right of the screen) you wish to use according to the endpoints instructions.
- Modify the paths to the data you wish to transfer. For this example, we will move a file from CADES Lustre storage to OLCF Atlas.
- Select the files or directories to transfer and begin the transfer using the directional arrows between endpoint, or click and drag files/folders between the two endpoint panels.
Ready to Learn More?