Using the RDSS Globus Endpoint

Quest and Kellogg Linux Cluster Downtime, December 14 - 18.

Quest, including the Quest Analytics Nodes, the Genomics Compute Cluster (GCC), the Kellogg Linux Cluster (KLC), and Quest OnDemand, will be unavailable for scheduled maintenance starting at 8 A.M. on Saturday, December 14, and ending approximately at 5 P.M. on Wednesday, December 18. During the maintenance window, you will not be able to login to Quest, Quest Analytics Nodes, the GCC, KLC, or Quest OnDemand submit new jobs, run jobs, or access files stored on Quest in any way including Globus. For details on this maintenance, please see the Status of University IT Services page.

Quest RHEL8 Pilot Environment - November 18.

Starting November 18, all Quest users are invited to test and run their workflows in a RHEL8 pilot environment to prepare for Quest moving completely to RHEL8 in March 2025. We invite researchers to provide us with feedback during the pilot by contacting the Research Computing and Data Services team at quest-help@northwestern.edu. The pilot environment will consist of 24 H100 GPU nodes and seventy-two CPU nodes, and it will expand with additional nodes through March 2025. Details on how to access this pilot environment will be published in a KB article on November 18.

The Northwestern Quest RDSS endpoint allows you to transfer data between RDSS/FSMResfiles and Quest. Currently, all data to/from RDSS must go through Quest due to current system architecture.

Setup

Prerequisites

To use the RDSS endpoint you must:

  1. Have a Quest Allocation: Follow the linked instructions to request a Quest allocation.
  2. Have access to an RDSS share or FSMResFiles folder: See this page on Managing access to RDSS Shares to find out how to request access. 
  3. Request access to RDSS/FSMResFiles via Globus by filling out the Request access to RDSS via Globus form

Once you are granted access, you need to give the Globus permission to access your share by mounting it on the data transfer node (qglobus12). 

Mounting your RDSS/FSMResFiles share to the data transfer node

Follow these steps to mount your share/folder to the data transfer node.

  1. Make sure you are either on campus, or are connected to GlobalProtect VPN. The qglobus12 node only accepts connections from certain IP Addresses and your connection will fail if you are not associated with a Northwestern IP address. 
  2. Then, open your terminal program of choice and connect to qglobus12 using the following command

ssh <netid>@qglobus12.ci.northwestern.edu

Where you replace <netid> with your Northwestern NetID. Enter your NetID password when prompted.

  1. This operation will automatically mount your RDSS/FSMResFiles share. The share will remain mounted after you end your terminal session. 

Note: Your share/folder will unmount if we have to restart qglobus12. If you ever get a "permission denied" error when accessing your files, follow these steps again to remount your share. If the error persists, email globus-help@northwestern.edu

Activating the endpoint

To activate the RDSS endpoint:

Open https://app.globus.org/file-manager in the browser and log in as a Northwestern user.

Then Search for the the RDSS endpoint "Northwestern Quest RDSS". 

Searching for and selecting RDSS endpoint in Globus file manager

When you select this endpoint, Globus will then ask you to authenticate using your Northwestern NetID and password

Then, you should see the /rdss/ directory, which contains folders with users' NetIDs. 

To navigate to access your RDSS/FSMResFiles content, navigate into the folder with your NetID as the name. Inside that folder, you will see a folder called "resfiles", "fsmresfiles", or "resfiles-audit", depending on where your allocation is located. Navigate into that folder. You should see the files contained in the directory that you requested access to in the Setup section.

We recommend bookmarking this folder to avoid having to navigate to this folder every time you initiate a file transfer. To create a bookmark, select the bookmark icon to the right of the path you would like to bookmark. The icon will appear yellow if the path is bookmarked. 

Bookmark a globus endpoint

Transferring files to/from RDSS/FSMResFiles

The Northwestern Quest RDSS endpoint only allows data transfers within RDSS and to the Northwestern Quest managed endpoint.

  • To transfer to Quest: No staging is necessary.
  • To transfer anywhere else: To transfer files from RDSS/FSMResfiles to any other endpoint, first transfer the data to Quest. Your Quest allocation must have enough storage space to accommodate the files that you're transferring. Then, you can transfer the files to any other Globus endpoint.

Extending RDSS/FSMResFiles endpoint activation

Northwestern Quest RDSS endpoint activation on Globus expires after 1 day by default. Once it expires, you must reauthenticate.

To extend the activation period: 

First, on the Globus website, find the Northwestern Quest RDSS Endpoint in the File Manager window.

Then, select the Manage Activation icon that looks like a power button at the bottom of the column between the file browser windows (red box)

 

extend rdss activation page, red box around button to open page

Click Extend Activation. Globus will ask you to reauthenticate with your Northwestern NetID

BEFORE YOU AUTHENTICATE, click Advanced under the password field, which will allow you to specify how long you would like the endpoint to remain activated up to 168 hours (7 days). Enter the amount of time you would like to extend the activation for in hours. 

If you would like to create a stable endpoint that does not expire, you can create a Shared Collection on the RDSS Endpoint. See Globus's How to Share Data Using Globus documentation for more information.

Troubleshooting

Here are some common issues with using the Quest RDSS Globus endpoint and how to resolve them.

I get a "permission denied" error when I try to access my files

Seeing a "permission denied" error when trying to access a folder on the Quest RDSS endpoint (below) means that Globus does not think you have permission to see the folder you're trying to access. 

Globus permission denied error

 

The two most common reasons for this are: 

Reason 1: You don't have access to the file or folder on RDSS/FSMResFiles

To check whether you have access on RDSS/FSMResFiles, connect to your shared drive on your computer and see if you have access. If you do not, 

  • RDSS users should contact the owner of your share. They can grant you access using our self-service tool.
  • FSMResFiles users should contact FSM-IT (fsmhelp@northwestern.edu) for access. 

Reason 2: Your RDSS share was unmounted from the Quest RDSS Globus node

Occasionally, the Quest RDSS Globus (qglobus12) node has to be restarted for system maintenance. Our current system architecture requires users to log in to the qglobus12 node to remount the share. Please see instructions in the "Mounting your RDSS/FSMResFiles share" section above for details.

If neither of these methods solve your problem, email globus-help@northwestern.edu and we will help you troubleshoot.

I can't get to the folder I need in the Globus interface

When we set up access to the Quest RDSS Globus endpoint for users, we have to choose a mountpoint, which determines what you see inside your NetID folder in Globus. For example, if you need access to multiple shares in the resfiles zone or multiple lab groups/departments in FSMResFiles, we set your mountpoint to the folder that contains these folders. If you can only see one of those shares/folders, we will have to change your mountpoint.

To request this change, email globus-help@northwestern.edu with the following information:

  • Your NetID
  • Where you want your mountpoint to be OR the filepaths you need to see in Globus. We can help you figure out the correct mountpoint if you give us all the locations you need access to.

My file transfer fails when I try to transfer files between RDSS and my personal endpoint

Our current infrastructure only allows transfers between the RDSS and Quest endpoints. You can transfer your data to Quest and then transfer your data to your personal endpoint or any other Globus endpoint. Do not use Quest to store sensitive data even temporarily. 

Email globus-help@northwestern.edu if you need help setting up this two-stage transfer, don't have enough storage space on Quest, or are working with sensitive data that cannot be stored on Quest. 

Need help?

Email globus-help@northwestern.edu for questions about using Globus.

Also see our Research Data Management Guide for links to all of our help articles.

Research Data Management Support at Northwestern University

Research computing data services partners with the Office for Research, University Libraries and Galter Health Sciences Library to provide research data management support throughout the research process. Please see the Research Data Management and Sharing page for more information

Was this helpful?
100% helpful - 1 review
Print Article

Related Articles (7)

An outline of advanced features available for Globus
How to identify the full path of your RDSS/FSMResfiles share to set up access to the RDSS Globus endpoint.
A description of Globus endpoints and collections available at Northwestern
How to initiate file transfers using the Globus data transfer tool
Overview of how to use Globus with links to how to articles.
How to set up and use the Quest Globus endpoint

Related Services / Offerings (1)

RDSS provides secure on-premise data storage for researchers.