Body
We strongly encourage all Quest users to review the summary of post-maintenance changes as these changes could immediately impact your workflow after the Quest March 2025 downtime.
This page summarizes guidelines and tools for managing allocations on Quest, Northwestern's high-performance compute (HPC) cluster. More information about general access allocations (including application forms) can be found here and details about purchased resources on Quest are described here.
Table of Contents
Below are the guidelines for who can fulfill certain roles when requesting an allocation, and what the capabilities of those roles are:
Allocation Principal Investigator:
The Allocation Principal Investigator (PI) is the owner of the allocation. Northwestern faculty and staff (e.g. Postdoctoral Fellow) are eligible to be listed as PIs for allocations. The PI's responsibilities include:
In certain circumstances exceptions can be made, allowing a graduate student to be the PI of the Quest allocation if there is no faculty or staff PI or adviser associated with the research project. Please reach out to quest-help@northwestern.edu if you have any questions.
Allocation Manager:
Unless otherwise specified, the Principal Investigator (PI) also serves as the Allocation Manager. Any Northwestern University researcher or educator, including graduate students, postdocs, faculty, and staff, can be listed as an allocation manager. Graduate students can list themselves as the Allocation Manager but must also include a sponsoring PI. Graduate students may only serve as the allocation manager for one active Quest allocation at a time. Allocation Manager’s responsibilities include:
Allocation Member:
Any Northwestern University researcher may request to join an allocation on the Quest computational cluster for the purpose of research. This includes Northwestern undergraduate students, graduate students, postdocs, faculty, and staff. Undergraduate students may use Quest but can only join an existing allocation associated with a sponsoring Allocation Manager, PI, or research project. As an Allocation Member, you can create directories and store files in the allocation, as well as submit jobs, within the parameters defined by the Allocation PI and Manager. Allocation Members can request data ownership changes with the direct approval from the Allocation PI.
Quest allocations are available for workshops and classroom use. Requests for such an allocation must be submitted by the instructor. Request a new allocation or manage an existing allocation on the Quest Allocations page.
Allocation Resources
A workshop and classroom allocation provides individual user accounts on Quest for all students added to the allocation. The group will have access to 2 TB of shared storage in a dedicated directory, and each student will have 80 GB in their individual home directories.
Allocation Use Cases
Workshop and classroom allocations are primarily for situations where students need significant computational resources (beyond what a typical laptop can handle) in order to complete their work.
While workshop and classroom allocations provide access to the Quest Analytics Nodes, the Quest Analytics Nodes can only accommodate moderate workloads. If you plan to assign work for students using the Quest Analytics Nodes that requires significant computational resources, please contact quest-help@northwestern.edu to discuss whether Quest can meet your needs.
If your primary interest in a Quest workshop and classroom allocation is as a backup for students with local software installation issues, consider having students use NUworkspace as a backup instead. NUworkspace provides many software programs, including R, Python, and Jupyter, through a virtual Windows desktop environment. For local software installation on student’s computers, R, RStudio, Python, and Jupyter are all freely available, and SAS provides a free University Edition for students, as well as free access to SAS OnDemand for Academics.
Instructor Responsibilities
Instructors are responsible for managing the allocation and student use of Quest. This includes:
-
Supplying NetIDs of students, instructors, and support staff who need access. You will be notified when accounts are created. Students will not be contacted by Northwestern IT directly regarding account creation.
-
Teaching students how to use the Quest resources necessary for the class or workshop. This includes logging in, transferring files to and from Quest, use of software, package installation and management, and, if needed, how to submit jobs. Instructors (or their TAs or staff) are responsible for fielding initial questions or issues. If you encounter an error with the system, please submit a help request to quest-help@northwestern.edu.
Timing and Duration
Requests for Workshop and Classroom allocations should be made at least one week prior to the start of the class.
The allocation will be active for the duration of the current (or upcoming) quarter; it will expire approximately three weeks after the end of the quarter. If you requested shared storage space as part of the allocation, files in it will be deleted when the allocation expires. Students and instructors who are not part of another active Quest allocation will have their accounts and files deleted when the classroom allocation expires.
Special Requests
If your classroom or workshop needs fall outside of the limits described above, please contact Research Computing and Data Services at quest-help@northwestern.edu to discuss options.
If your class or workshop requires students to run batch jobs on Quest via the scheduler, Research Computing and Data Services staff can work with you to help students get started.
Add Users to an Allocation
Use the Join an Existing Allocation form. The allocation's manager will be notified via email of any user requesting to join the allocation, which they need to either accept or deny. Individuals added to an allocation will have access to the /projects/<allocationID> directory, but they will have their own home directory.
Renew an Existing Allocation or Create a New Allocation
Use the appropriate Research Allocation Original or Renewal Request form for your allocation type (I or II).
List Allocation Members
Display all current members of the allocation:
module load utilities
grouplist <allocationID>
Checking Allocation Resources
You can find out your allocations by using groups command on Quest. The output of this command will include your NetID and the IDs of allocations that you belong.
To check the storage use of /projects/<allocationID> and allocation expiration:
checkproject <allocationID>
Example Output
$ checkproject <allocationID>
====================================
Reporting for project <allocationID>
------------------------------------
1 GB in 4623 files (0% of 1000 GB quota)
Allocation Type: Allocation I
Expiration Date: 2022-12-01
Status: ACTIVE
Compute and storage allocation - when status is ACTIVE, this allocation has compute node access and can submit jobs
------------------------------------
====================================
How to obtain information on full access/buy-in allocation resources on Quest.
Users with full access, also referred to as buy-in allocations, can view information about their nodes, queues, and allocation members.
Full access allocation IDs usually start with the letter b; e.g. b1002. Use your allocation ID in place of <allocationID> in all commands below, without the surrounding <>.
List Allocation Members
Display all current members of the allocation:
module load utilities
grouplist <allocationID>
Status of Allocation Jobs in Queue
Display the the status of all jobs submitted to an allocation
squeue –A <allocationID>
Example Output
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
79872 b1002 hisat2-b ghi678 PD 0:00 1 (None)
79872 b1002 bash def345 R 12:42 1 qgpu8014
79851 b1002 rv2rg8z0 abc123 R 10:22:46 2 qnode[8010-8011]
79846 b1002 vasp abc123 R 11:16:41 5 qnode[6062-6066]
Status of Allocation Nodes
The sinfo command will give you information about state of a particular account's compute nodes. For full access allocations which are using "buyin" partition (i.e. queue), nodes associated with the allocation can be listed as below:
sinfo -N -p <allocationID>
For full access allocations which have multiple partitions (i.e. queues) such as b1042, b0194 or b1095, nodes associated with the specific queue can be listed as below:
sinfo -N -p <partitionID>
Individual Node Status
After you identified your allocation's and/or partition's nodes using sinfo you can use the following command to get detailed information about a specific node:
scontrol show node <nodeID>
Example
scontrol show node qnode5004
If the above commands don't work for your allocation, contact quest-help@northwestern.edu for assistance.
Why are my jobs running on nodes other than those reserved for my allocation?
If you have access to more than one allocation or queue (i.e. partition) on Quest, you may have defined a different allocation or queue in your job submission script. Please make sure that the intended allocation or queue have been setup correctly in your job submission script.